Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
2026-08-09T01:45:09Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:45:09Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260809 01:45:09.373029 10273 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
/tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.126:42157
--webserver_interface=127.10.8.126
--webserver_port=0
--builtin_ntp_servers=127.10.8.84:40957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.8.126:42157,127.10.8.125:42063,127.10.8.124:33929
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260809 01:45:09.678960 10289 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:45:09.679607 10289 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:45:09.680068 10289 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 01:45:09.710546 10289 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:45:09.710804 10289 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:45:09.711163 10289 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:45:09.711436 10289 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:45:09.746518 10289 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.8.84:40957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.8.126:42157,127.10.8.125:42063,127.10.8.124:33929
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.126:42157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.8.126
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 01:43:22 UTC on e7f111948823
build id 13251
TSAN enabled
I20260809 01:45:09.748507 10289 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:45:09.750543 10289 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 01:45:09.766929 10300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:45:09.767344 10301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:45:11.059950 10302 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
W20260809 01:45:11.061802 10303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:45:11.065636 10289 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.301s user 0.496s sys 0.804s
W20260809 01:45:11.066048 10289 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.302s user 0.497s sys 0.804s
I20260809 01:45:11.066334 10289 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260809 01:45:11.068040 10289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:45:11.072073 10289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:45:11.073601 10289 hybrid_clock.cc:648] HybridClock initialized: now 1786239911073573 us; error 46 us; skew 500 ppm
I20260809 01:45:11.075223 10289 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:45:11.085217 10289 webserver.cc:533] Webserver started at http://127.10.8.126:45089/ using document root <none> and password file <none>
I20260809 01:45:11.087036 10289 fs_manager.cc:362] Metadata directory not provided
I20260809 01:45:11.087388 10289 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:45:11.088209 10289 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:45:11.095957 10289 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/data/instance:
uuid: "09a73ee8e8ec496a984efcab27715506"
format_stamp: "Formatted at 2026-08-09 01:45:11 on dist-test-slave-zr1t"
I20260809 01:45:11.097795 10289 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/wal/instance:
uuid: "09a73ee8e8ec496a984efcab27715506"
format_stamp: "Formatted at 2026-08-09 01:45:11 on dist-test-slave-zr1t"
I20260809 01:45:11.107249 10289 fs_manager.cc:696] Time spent creating directory manager: real 0.009s user 0.008s sys 0.000s
I20260809 01:45:11.115916 10314 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:45:11.117596 10289 fs_manager.cc:730] Time spent opening block manager: real 0.006s user 0.002s sys 0.004s
I20260809 01:45:11.117988 10289 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/data,/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/wal
uuid: "09a73ee8e8ec496a984efcab27715506"
format_stamp: "Formatted at 2026-08-09 01:45:11 on dist-test-slave-zr1t"
I20260809 01:45:11.118377 10289 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:45:11.210959 10289 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 01:45:11.212637 10289 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:45:11.213155 10289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:45:11.293417 10289 rpc_server.cc:307] RPC server started. Bound to: 127.10.8.126:42157
I20260809 01:45:11.293599 10399 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.8.126:42157 every 8 connection(s)
I20260809 01:45:11.296633 10289 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/data/info.pb
I20260809 01:45:11.305897 10273 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu as pid 10289
I20260809 01:45:11.306378 10273 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-0/wal/instance
I20260809 01:45:11.306450 10400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:11.309947 10273 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
/tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.125:42063
--webserver_interface=127.10.8.125
--webserver_port=0
--builtin_ntp_servers=127.10.8.84:40957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.8.126:42157,127.10.8.125:42063,127.10.8.124:33929
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260809 01:45:11.321765 10400 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:45:11.359117 10400 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } has no permanent_uuid. Determining permanent_uuid...
W20260809 01:45:11.361850 10320 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
W20260809 01:45:11.366546 10400 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.125:42063: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
I20260809 01:45:11.416118 10400 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } attempt: 1
W20260809 01:45:11.420950 10400 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.125:42063: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
I20260809 01:45:11.489457 10400 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } attempt: 2
W20260809 01:45:11.493845 10400 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.125:42063: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
I20260809 01:45:11.585373 10400 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } attempt: 3
W20260809 01:45:11.590588 10400 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.125:42063: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
W20260809 01:45:11.624878 10404 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:45:11.625430 10404 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:45:11.625797 10404 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 01:45:11.656414 10404 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:45:11.656701 10404 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:45:11.656903 10404 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:45:11.657099 10404 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:45:11.692636 10404 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.8.84:40957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.8.126:42157,127.10.8.125:42063,127.10.8.124:33929
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.125:42063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.8.125
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 01:43:22 UTC on e7f111948823
build id 13251
TSAN enabled
I20260809 01:45:11.693907 10404 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:45:11.695428 10404 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 01:45:11.706638 10418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:45:11.734119 10400 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } attempt: 4
W20260809 01:45:11.742025 10400 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.125:42063: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
I20260809 01:45:12.041647 10400 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } attempt: 5
W20260809 01:45:12.049605 10400 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.125:42063: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
I20260809 01:45:12.597185 10400 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } attempt: 6
W20260809 01:45:12.633630 10400 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.125:42063: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
W20260809 01:45:13.110735 10416 debug-util.cc:398] Leaking SignalData structure 0x7b080002ede0 after lost signal to thread 10404
W20260809 01:45:11.707201 10420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:45:13.488879 10416 kernel_stack_watchdog.cc:198] Thread 10404 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 400ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260809 01:45:13.489892 10404 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.783s user 0.551s sys 1.230s
W20260809 01:45:13.490257 10404 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.784s user 0.551s sys 1.230s
W20260809 01:45:13.494613 10422 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1787 milliseconds
W20260809 01:45:13.495334 10423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:45:13.495381 10404 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260809 01:45:13.496622 10404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:45:13.498988 10404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:45:13.500339 10404 hybrid_clock.cc:648] HybridClock initialized: now 1786239913500300 us; error 62 us; skew 500 ppm
I20260809 01:45:13.501106 10404 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:45:13.507212 10404 webserver.cc:533] Webserver started at http://127.10.8.125:46803/ using document root <none> and password file <none>
I20260809 01:45:13.508093 10404 fs_manager.cc:362] Metadata directory not provided
I20260809 01:45:13.508301 10404 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:45:13.508684 10404 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:45:13.512862 10404 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/data/instance:
uuid: "71bc72deb1d2402cabd817dfe3b27e88"
format_stamp: "Formatted at 2026-08-09 01:45:13 on dist-test-slave-zr1t"
I20260809 01:45:13.514024 10404 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/wal/instance:
uuid: "71bc72deb1d2402cabd817dfe3b27e88"
format_stamp: "Formatted at 2026-08-09 01:45:13 on dist-test-slave-zr1t"
I20260809 01:45:13.521706 10404 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260809 01:45:13.527266 10438 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:45:13.528462 10404 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260809 01:45:13.528797 10404 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/data,/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/wal
uuid: "71bc72deb1d2402cabd817dfe3b27e88"
format_stamp: "Formatted at 2026-08-09 01:45:13 on dist-test-slave-zr1t"
I20260809 01:45:13.529141 10404 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:45:13.584088 10404 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 01:45:13.585669 10404 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:45:13.586125 10404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:45:13.659905 10404 rpc_server.cc:307] RPC server started. Bound to: 127.10.8.125:42063
I20260809 01:45:13.659987 10517 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.8.125:42063 every 8 connection(s)
I20260809 01:45:13.662663 10404 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/data/info.pb
I20260809 01:45:13.667416 10273 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu as pid 10404
I20260809 01:45:13.667960 10273 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/wal/instance
I20260809 01:45:13.669654 10518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:13.671257 10273 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
/tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.124:33929
--webserver_interface=127.10.8.124
--webserver_port=0
--builtin_ntp_servers=127.10.8.84:40957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.8.126:42157,127.10.8.125:42063,127.10.8.124:33929
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260809 01:45:13.682842 10518 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:45:13.694190 10400 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } attempt: 7
I20260809 01:45:13.711936 10518 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:45:13.719085 10400 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } has no permanent_uuid. Determining permanent_uuid...
W20260809 01:45:13.726181 10400 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:33929: Network error: Client connection negotiation failed: client connection to 127.10.8.124:33929: connect: Connection refused (error 111)
I20260809 01:45:13.726521 10518 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } has no permanent_uuid. Determining permanent_uuid...
W20260809 01:45:13.729202 10441 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.124:33929: connect: Connection refused (error 111)
W20260809 01:45:13.734362 10518 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:33929: Network error: Client connection negotiation failed: client connection to 127.10.8.124:33929: connect: Connection refused (error 111)
I20260809 01:45:13.783896 10518 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } attempt: 1
I20260809 01:45:13.784657 10400 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } attempt: 1
W20260809 01:45:13.790256 10518 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:33929: Network error: Client connection negotiation failed: client connection to 127.10.8.124:33929: connect: Connection refused (error 111)
W20260809 01:45:13.790498 10400 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:33929: Network error: Client connection negotiation failed: client connection to 127.10.8.124:33929: connect: Connection refused (error 111)
I20260809 01:45:13.858681 10518 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } attempt: 2
W20260809 01:45:13.862995 10518 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:33929: Network error: Client connection negotiation failed: client connection to 127.10.8.124:33929: connect: Connection refused (error 111)
I20260809 01:45:13.871874 10400 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } attempt: 2
W20260809 01:45:13.875893 10400 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:33929: Network error: Client connection negotiation failed: client connection to 127.10.8.124:33929: connect: Connection refused (error 111)
I20260809 01:45:13.954362 10518 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } attempt: 3
W20260809 01:45:13.961426 10518 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:33929: Network error: Client connection negotiation failed: client connection to 127.10.8.124:33929: connect: Connection refused (error 111)
I20260809 01:45:13.961282 10400 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } attempt: 3
W20260809 01:45:13.972051 10400 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:33929: Network error: Client connection negotiation failed: client connection to 127.10.8.124:33929: connect: Connection refused (error 111)
W20260809 01:45:14.058866 10520 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:45:14.059545 10520 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:45:14.059957 10520 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 01:45:14.093194 10520 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:45:14.093478 10520 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:45:14.093706 10520 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:45:14.093910 10520 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:45:14.105021 10518 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } attempt: 4
W20260809 01:45:14.112334 10518 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:33929: Network error: Client connection negotiation failed: client connection to 127.10.8.124:33929: connect: Connection refused (error 111)
I20260809 01:45:14.112610 10400 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } attempt: 4
W20260809 01:45:14.119704 10400 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:33929: Network error: Client connection negotiation failed: client connection to 127.10.8.124:33929: connect: Connection refused (error 111)
I20260809 01:45:14.134835 10520 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.8.84:40957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.8.126:42157,127.10.8.125:42063,127.10.8.124:33929
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.124:33929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.8.124
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 01:43:22 UTC on e7f111948823
build id 13251
TSAN enabled
I20260809 01:45:14.136094 10520 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:45:14.137780 10520 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 01:45:14.155138 10543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:45:14.403290 10400 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } attempt: 5
W20260809 01:45:14.411635 10400 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:33929: Network error: Client connection negotiation failed: client connection to 127.10.8.124:33929: connect: Connection refused (error 111)
I20260809 01:45:14.411923 10518 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } attempt: 5
W20260809 01:45:14.422996 10518 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:33929: Network error: Client connection negotiation failed: client connection to 127.10.8.124:33929: connect: Connection refused (error 111)
I20260809 01:45:14.964349 10400 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } attempt: 6
I20260809 01:45:14.970686 10518 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } attempt: 6
W20260809 01:45:14.989490 10400 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:33929: Network error: Client connection negotiation failed: client connection to 127.10.8.124:33929: connect: Connection refused (error 111)
W20260809 01:45:14.990978 10518 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:33929: Network error: Client connection negotiation failed: client connection to 127.10.8.124:33929: connect: Connection refused (error 111)
W20260809 01:45:14.160660 10546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:45:15.544842 10550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:45:15.549516 10520 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.385s user 0.004s sys 0.014s
W20260809 01:45:15.549822 10520 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.386s user 0.008s sys 0.014s
I20260809 01:45:15.550087 10520 server_base.cc:1061] running on GCE node
I20260809 01:45:15.551139 10520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:45:15.555528 10520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:45:15.559263 10520 hybrid_clock.cc:648] HybridClock initialized: now 1786239915559225 us; error 56 us; skew 500 ppm
I20260809 01:45:15.560093 10520 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:45:15.568949 10520 webserver.cc:533] Webserver started at http://127.10.8.124:33525/ using document root <none> and password file <none>
I20260809 01:45:15.570070 10520 fs_manager.cc:362] Metadata directory not provided
I20260809 01:45:15.570333 10520 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:45:15.570864 10520 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:45:15.575407 10520 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/data/instance:
uuid: "629f002048f64491afdd57f72a466e59"
format_stamp: "Formatted at 2026-08-09 01:45:15 on dist-test-slave-zr1t"
I20260809 01:45:15.576519 10520 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/wal/instance:
uuid: "629f002048f64491afdd57f72a466e59"
format_stamp: "Formatted at 2026-08-09 01:45:15 on dist-test-slave-zr1t"
I20260809 01:45:15.584671 10520 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.007s sys 0.003s
I20260809 01:45:15.591341 10572 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:45:15.592665 10520 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.003s sys 0.003s
I20260809 01:45:15.592994 10520 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/data,/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/wal
uuid: "629f002048f64491afdd57f72a466e59"
format_stamp: "Formatted at 2026-08-09 01:45:15 on dist-test-slave-zr1t"
I20260809 01:45:15.593447 10520 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:45:15.697475 10520 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 01:45:15.700066 10520 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:45:15.700680 10520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:45:15.977041 10520 rpc_server.cc:307] RPC server started. Bound to: 127.10.8.124:33929
I20260809 01:45:15.977388 10651 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.8.124:33929 every 8 connection(s)
I20260809 01:45:15.980077 10520 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/data/info.pb
I20260809 01:45:15.989406 10273 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu as pid 10520
I20260809 01:45:15.990010 10273 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-2/wal/instance
I20260809 01:45:15.993508 10653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:16.023103 10400 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } attempt: 7
I20260809 01:45:16.038453 10653 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:45:16.051549 10518 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } attempt: 7
I20260809 01:45:16.097008 10653 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:45:16.110291 10400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506: Bootstrap starting.
I20260809 01:45:16.125099 10653 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:45:16.126004 10400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:16.131139 10518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: Bootstrap starting.
I20260809 01:45:16.132411 10400 log.cc:826] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506: Log is configured to *not* fsync() on all Append() calls
I20260809 01:45:16.146549 10400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506: No bootstrap required, opened a new log
I20260809 01:45:16.149199 10518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:16.153713 10518 log.cc:826] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: Log is configured to *not* fsync() on all Append() calls
I20260809 01:45:16.162340 10518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: No bootstrap required, opened a new log
I20260809 01:45:16.171092 10653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Bootstrap starting.
I20260809 01:45:16.184837 10653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:16.186270 10400 raft_consensus.cc:359] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:16.187359 10400 raft_consensus.cc:385] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:16.187805 10400 raft_consensus.cc:740] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09a73ee8e8ec496a984efcab27715506, State: Initialized, Role: FOLLOWER
I20260809 01:45:16.188175 10653 log.cc:826] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Log is configured to *not* fsync() on all Append() calls
I20260809 01:45:16.189447 10400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:16.200557 10653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: No bootstrap required, opened a new log
I20260809 01:45:16.199971 10680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:16.200971 10680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:45:16.202553 10518 raft_consensus.cc:359] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:16.203438 10518 raft_consensus.cc:385] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:16.203799 10518 raft_consensus.cc:740] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bc72deb1d2402cabd817dfe3b27e88, State: Initialized, Role: FOLLOWER
I20260809 01:45:16.205047 10518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:16.215191 10400 sys_catalog.cc:565] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:45:16.215001 10681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:16.217064 10681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:45:16.221002 10518 sys_catalog.cc:565] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 01:45:16.226899 10326 tablet.cc:2406] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:45:16.226938 10680 raft_consensus.cc:493] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:16.228052 10680 raft_consensus.cc:515] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:16.239853 10680 leader_election.cc:290] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:16.244114 10480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09a73ee8e8ec496a984efcab27715506" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bc72deb1d2402cabd817dfe3b27e88" is_pre_election: true
I20260809 01:45:16.245935 10480 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09a73ee8e8ec496a984efcab27715506 in term 0.
I20260809 01:45:16.248723 10320 leader_election.cc:304] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09a73ee8e8ec496a984efcab27715506, 71bc72deb1d2402cabd817dfe3b27e88; no voters:
I20260809 01:45:16.250408 10680 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:16.251019 10680 raft_consensus.cc:493] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:16.251636 10680 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:16.306010 10680 raft_consensus.cc:515] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:16.321110 10680 leader_election.cc:290] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 1 election: Requested vote from peers 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:16.322624 10480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09a73ee8e8ec496a984efcab27715506" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bc72deb1d2402cabd817dfe3b27e88"
I20260809 01:45:16.323544 10480 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:16.333312 10653 raft_consensus.cc:359] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:16.334859 10653 raft_consensus.cc:385] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:16.335779 10653 raft_consensus.cc:740] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629f002048f64491afdd57f72a466e59, State: Initialized, Role: FOLLOWER
I20260809 01:45:16.337880 10653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:16.348286 10480 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09a73ee8e8ec496a984efcab27715506 in term 1.
W20260809 01:45:16.349053 10447 tablet.cc:2406] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:45:16.350034 10320 leader_election.cc:304] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09a73ee8e8ec496a984efcab27715506, 71bc72deb1d2402cabd817dfe3b27e88; no voters:
I20260809 01:45:16.350907 10692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:16.353376 10692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:45:16.354025 10680 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:16.357383 10692 raft_consensus.cc:493] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:16.358394 10692 raft_consensus.cc:515] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:16.362432 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09a73ee8e8ec496a984efcab27715506" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59"
I20260809 01:45:16.363552 10612 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:16.366258 10613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09a73ee8e8ec496a984efcab27715506" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:45:16.361379 10653 sys_catalog.cc:565] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:45:16.371248 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:16.375530 10480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bc72deb1d2402cabd817dfe3b27e88" is_pre_election: true
I20260809 01:45:16.376863 10480 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 629f002048f64491afdd57f72a466e59 in current term 1: Already voted for candidate 09a73ee8e8ec496a984efcab27715506 in this term.
I20260809 01:45:16.378669 10680 raft_consensus.cc:697] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 1 LEADER]: Becoming Leader. State: Replica: 09a73ee8e8ec496a984efcab27715506, State: Running, Role: LEADER
I20260809 01:45:16.381243 10692 leader_election.cc:290] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063)
I20260809 01:45:16.380484 10680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:16.398161 10612 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09a73ee8e8ec496a984efcab27715506 in term 1.
I20260809 01:45:16.428193 10697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09a73ee8e8ec496a984efcab27715506. Latest consensus state: current_term: 1 leader_uuid: "09a73ee8e8ec496a984efcab27715506" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:16.429307 10697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [sys.catalog]: This master's current role is: LEADER
I20260809 01:45:16.430929 10722 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 01:45:16.434401 10576 leader_election.cc:304] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 629f002048f64491afdd57f72a466e59; no voters: 09a73ee8e8ec496a984efcab27715506, 71bc72deb1d2402cabd817dfe3b27e88
I20260809 01:45:16.436425 10692 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:16.454617 10724 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
W20260809 01:45:16.468672 10583 tablet.cc:2406] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:45:16.438530 10404 master_runner.cc:187] Error getting master registration for 127.10.8.126:42157: OK, instance_id { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" instance_seqno: 1786239911271853 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 01:45:16.512342 10404 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:45:16.513072 10724 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 01:45:16.521816 10739 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 01:45:16.527237 10520 master_runner.cc:187] Error getting master registration for 127.10.8.126:42157: OK, instance_id { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" instance_seqno: 1786239911271853 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 01:45:16.574539 10520 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:45:16.585449 10612 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Refusing update from remote peer 09a73ee8e8ec496a984efcab27715506: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:45:16.586059 10480 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Refusing update from remote peer 09a73ee8e8ec496a984efcab27715506: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:45:16.587342 10697 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:16.588182 10680 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:16.621120 10681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09a73ee8e8ec496a984efcab27715506. Latest consensus state: current_term: 1 leader_uuid: "09a73ee8e8ec496a984efcab27715506" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:16.622073 10681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:45:16.631892 10692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09a73ee8e8ec496a984efcab27715506. Latest consensus state: current_term: 1 leader_uuid: "09a73ee8e8ec496a984efcab27715506" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:16.633092 10692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:45:16.634684 10746 mvcc.cc:204] Tried to move back new op lower bound from 7316438698311733248 to 7316438697581670400. Current Snapshot: MvccSnapshot[applied={T|T < 7316438698311733248}]
I20260809 01:45:16.642974 10681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09a73ee8e8ec496a984efcab27715506" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:16.644076 10681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:45:16.651656 10692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09a73ee8e8ec496a984efcab27715506" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:16.652539 10692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:45:16.652230 10697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09a73ee8e8ec496a984efcab27715506" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:16.653090 10697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [sys.catalog]: This master's current role is: LEADER
I20260809 01:45:16.653872 10697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09a73ee8e8ec496a984efcab27715506" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:16.654999 10697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [sys.catalog]: This master's current role is: LEADER
I20260809 01:45:16.656388 10724 catalog_manager.cc:1383] Generated new cluster ID: 465fa00c323a404c8f72f4f6eb53200b
I20260809 01:45:16.656805 10724 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 01:45:16.718165 10724 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 01:45:16.720877 10724 catalog_manager.cc:1540] Loading token signing keys...
I20260809 01:45:16.773231 10724 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506: Generated new TSK 0
I20260809 01:45:16.775089 10724 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 01:45:16.814453 10273 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
/tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.8.65:0
--local_ip_for_outbound_sockets=127.10.8.65
--webserver_interface=127.10.8.65
--webserver_port=0
--tserver_master_addrs=127.10.8.126:42157,127.10.8.125:42063,127.10.8.124:33929
--builtin_ntp_servers=127.10.8.84:40957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260809 01:45:17.267058 10763 raft_consensus.cc:493] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 09a73ee8e8ec496a984efcab27715506)
I20260809 01:45:17.267689 10763 raft_consensus.cc:515] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:17.271309 10763 leader_election.cc:290] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:17.271734 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:17.273249 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:45:17.274976 10441 leader_election.cc:304] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bc72deb1d2402cabd817dfe3b27e88; no voters: 09a73ee8e8ec496a984efcab27715506, 629f002048f64491afdd57f72a466e59
I20260809 01:45:17.276198 10763 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 01:45:17.346835 10395 debug-util.cc:398] Leaking SignalData structure 0x7b080007bb40 after lost signal to thread 10290
W20260809 01:45:17.347590 10395 debug-util.cc:398] Leaking SignalData structure 0x7b0800078fc0 after lost signal to thread 10399
W20260809 01:45:17.412796 10762 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:45:17.413661 10762 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:45:17.414696 10762 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 01:45:17.457732 10722 catalog_manager.cc:1295] Loaded cluster ID: 465fa00c323a404c8f72f4f6eb53200b
I20260809 01:45:17.463095 10722 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: loading cluster ID for follower catalog manager: success
I20260809 01:45:17.478322 10722 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: acquiring CA information for follower catalog manager: success
W20260809 01:45:17.485159 10762 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:45:17.486925 10762 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.8.65
I20260809 01:45:17.496724 10722 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 01:45:17.542024 10739 catalog_manager.cc:1295] Loaded cluster ID: 465fa00c323a404c8f72f4f6eb53200b
I20260809 01:45:17.542416 10739 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: loading cluster ID for follower catalog manager: success
I20260809 01:45:17.550088 10739 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: acquiring CA information for follower catalog manager: success
I20260809 01:45:17.556384 10739 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 01:45:17.588639 10762 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.8.84:40957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.8.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.8.65
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.8.126:42157,127.10.8.125:42063,127.10.8.124:33929
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.8.65
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 01:43:22 UTC on e7f111948823
build id 13251
TSAN enabled
I20260809 01:45:17.590536 10762 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:45:17.592829 10762 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 01:45:17.634333 10775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:45:17.682751 10783 raft_consensus.cc:493] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 09a73ee8e8ec496a984efcab27715506)
I20260809 01:45:17.683832 10783 raft_consensus.cc:515] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:17.688205 10783 leader_election.cc:290] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063)
I20260809 01:45:17.688592 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:17.691485 10480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "71bc72deb1d2402cabd817dfe3b27e88" is_pre_election: true
I20260809 01:45:17.694730 10576 leader_election.cc:304] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 629f002048f64491afdd57f72a466e59; no voters: 09a73ee8e8ec496a984efcab27715506, 71bc72deb1d2402cabd817dfe3b27e88
I20260809 01:45:17.696122 10783 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:45:18.352732 10787 raft_consensus.cc:493] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 09a73ee8e8ec496a984efcab27715506)
I20260809 01:45:18.354147 10787 raft_consensus.cc:515] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:18.367425 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:18.373426 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:45:18.376518 10441 leader_election.cc:304] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bc72deb1d2402cabd817dfe3b27e88; no voters: 09a73ee8e8ec496a984efcab27715506, 629f002048f64491afdd57f72a466e59
I20260809 01:45:18.381505 10787 leader_election.cc:290] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:18.383000 10787 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:45:19.062973 10790 raft_consensus.cc:493] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 09a73ee8e8ec496a984efcab27715506)
I20260809 01:45:19.064604 10790 raft_consensus.cc:515] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:19.077824 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:19.078114 10790 leader_election.cc:290] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063)
I20260809 01:45:19.082557 10480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "71bc72deb1d2402cabd817dfe3b27e88" is_pre_election: true
I20260809 01:45:19.085958 10581 leader_election.cc:304] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 629f002048f64491afdd57f72a466e59; no voters: 09a73ee8e8ec496a984efcab27715506, 71bc72deb1d2402cabd817dfe3b27e88
I20260809 01:45:19.087209 10790 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 01:45:17.653636 10776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:45:17.686793 10762 server_base.cc:1061] running on GCE node
W20260809 01:45:17.659487 10781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:45:19.691203 10762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:45:19.698249 10762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:45:19.714054 10762 hybrid_clock.cc:648] HybridClock initialized: now 1786239919713954 us; error 124 us; skew 500 ppm
I20260809 01:45:19.715229 10762 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:45:19.733064 10797 raft_consensus.cc:493] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 09a73ee8e8ec496a984efcab27715506)
I20260809 01:45:19.745740 10797 raft_consensus.cc:515] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:19.752660 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:19.756417 10797 leader_election.cc:290] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:19.756974 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:45:19.757756 10762 webserver.cc:533] Webserver started at http://127.10.8.65:40729/ using document root <none> and password file <none>
I20260809 01:45:19.758404 10441 leader_election.cc:304] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bc72deb1d2402cabd817dfe3b27e88; no voters: 09a73ee8e8ec496a984efcab27715506, 629f002048f64491afdd57f72a466e59
I20260809 01:45:19.758975 10762 fs_manager.cc:362] Metadata directory not provided
I20260809 01:45:19.759214 10762 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:45:19.759431 10797 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:45:19.759886 10762 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:45:19.774358 10762 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/data/instance:
uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
format_stamp: "Formatted at 2026-08-09 01:45:19 on dist-test-slave-zr1t"
I20260809 01:45:19.776124 10762 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/instance:
uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
format_stamp: "Formatted at 2026-08-09 01:45:19 on dist-test-slave-zr1t"
I20260809 01:45:19.795601 10762 fs_manager.cc:696] Time spent creating directory manager: real 0.019s user 0.008s sys 0.000s
I20260809 01:45:19.816239 10803 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:45:19.817691 10762 fs_manager.cc:730] Time spent opening block manager: real 0.017s user 0.007s sys 0.001s
I20260809 01:45:19.818367 10762 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/data,/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal
uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
format_stamp: "Formatted at 2026-08-09 01:45:19 on dist-test-slave-zr1t"
I20260809 01:45:19.819020 10762 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:45:19.976754 10809 raft_consensus.cc:493] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 09a73ee8e8ec496a984efcab27715506)
I20260809 01:45:19.977391 10809 raft_consensus.cc:515] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:19.980289 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:19.981189 10809 leader_election.cc:290] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063)
I20260809 01:45:19.982748 10480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "71bc72deb1d2402cabd817dfe3b27e88" is_pre_election: true
I20260809 01:45:19.984277 10581 leader_election.cc:304] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 629f002048f64491afdd57f72a466e59; no voters: 09a73ee8e8ec496a984efcab27715506, 71bc72deb1d2402cabd817dfe3b27e88
I20260809 01:45:19.993705 10809 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:45:20.019124 10762 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 01:45:20.028155 10762 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:45:20.028899 10762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:45:20.045843 10762 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 01:45:20.057032 10762 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:45:20.057420 10762 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 01:45:20.057858 10762 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 01:45:20.058151 10762 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:45:20.580909 10886 raft_consensus.cc:493] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 09a73ee8e8ec496a984efcab27715506)
I20260809 01:45:20.581877 10886 raft_consensus.cc:515] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:20.593413 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:20.598901 10480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "71bc72deb1d2402cabd817dfe3b27e88" is_pre_election: true
I20260809 01:45:20.601554 10581 leader_election.cc:304] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 629f002048f64491afdd57f72a466e59; no voters: 09a73ee8e8ec496a984efcab27715506, 71bc72deb1d2402cabd817dfe3b27e88
I20260809 01:45:20.606627 10886 leader_election.cc:290] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063)
I20260809 01:45:20.608119 10886 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:45:20.784247 10906 raft_consensus.cc:493] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 09a73ee8e8ec496a984efcab27715506)
I20260809 01:45:20.785058 10906 raft_consensus.cc:515] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:20.788964 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:20.794425 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:45:20.797101 10441 leader_election.cc:304] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bc72deb1d2402cabd817dfe3b27e88; no voters: 09a73ee8e8ec496a984efcab27715506, 629f002048f64491afdd57f72a466e59
I20260809 01:45:20.803112 10906 leader_election.cc:290] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:20.804368 10906 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:45:21.162158 10953 raft_consensus.cc:493] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 09a73ee8e8ec496a984efcab27715506)
I20260809 01:45:21.162798 10953 raft_consensus.cc:515] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:21.172649 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:21.176007 10953 leader_election.cc:290] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063)
I20260809 01:45:21.177816 10480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "71bc72deb1d2402cabd817dfe3b27e88" is_pre_election: true
I20260809 01:45:21.180001 10581 leader_election.cc:304] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 629f002048f64491afdd57f72a466e59; no voters: 09a73ee8e8ec496a984efcab27715506, 71bc72deb1d2402cabd817dfe3b27e88
I20260809 01:45:21.180923 10953 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:45:21.288491 10762 rpc_server.cc:307] RPC server started. Bound to: 127.10.8.65:41349
I20260809 01:45:21.288770 10983 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.8.65:41349 every 8 connection(s)
I20260809 01:45:21.291728 10762 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/data/info.pb
I20260809 01:45:21.296512 10273 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu as pid 10762
I20260809 01:45:21.297873 10273 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/instance
I20260809 01:45:21.331787 10273 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
/tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.8.66:0
--local_ip_for_outbound_sockets=127.10.8.66
--webserver_interface=127.10.8.66
--webserver_port=0
--tserver_master_addrs=127.10.8.126:42157,127.10.8.125:42063,127.10.8.124:33929
--builtin_ntp_servers=127.10.8.84:40957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260809 01:45:21.453100 10984 heartbeater.cc:344] Connected to a master server at 127.10.8.124:33929
I20260809 01:45:21.453768 10984 heartbeater.cc:461] Registering TS with master...
I20260809 01:45:21.455915 10984 heartbeater.cc:507] Master 127.10.8.124:33929 requested a full tablet report, sending...
I20260809 01:45:21.462049 10599 ts_manager.cc:194] Registered new tserver with Master: b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:21.467008 10986 heartbeater.cc:344] Connected to a master server at 127.10.8.125:42063
I20260809 01:45:21.467440 10986 heartbeater.cc:461] Registering TS with master...
I20260809 01:45:21.468559 10986 heartbeater.cc:507] Master 127.10.8.125:42063 requested a full tablet report, sending...
I20260809 01:45:21.475555 10464 ts_manager.cc:194] Registered new tserver with Master: b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:21.476516 10993 heartbeater.cc:344] Connected to a master server at 127.10.8.126:42157
I20260809 01:45:21.476959 10993 heartbeater.cc:461] Registering TS with master...
I20260809 01:45:21.478348 10993 heartbeater.cc:507] Master 127.10.8.126:42157 requested a full tablet report, sending...
I20260809 01:45:21.481940 10344 ts_manager.cc:194] Registered new tserver with Master: b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:21.486367 10344 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.10.8.65:37329
I20260809 01:45:21.774645 11009 raft_consensus.cc:493] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 09a73ee8e8ec496a984efcab27715506)
I20260809 01:45:21.775104 11009 raft_consensus.cc:515] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:21.776944 11009 leader_election.cc:290] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063)
I20260809 01:45:21.777813 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:21.779095 10480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "71bc72deb1d2402cabd817dfe3b27e88" is_pre_election: true
I20260809 01:45:21.782538 10581 leader_election.cc:304] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 629f002048f64491afdd57f72a466e59; no voters: 09a73ee8e8ec496a984efcab27715506, 71bc72deb1d2402cabd817dfe3b27e88
I20260809 01:45:21.783891 11009 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 01:45:21.899536 10994 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:45:21.900074 10994 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:45:21.900658 10994 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 01:45:21.933501 10994 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:45:21.934479 10994 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.8.66
I20260809 01:45:21.974929 10994 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.8.84:40957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.8.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.8.66
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.8.126:42157,127.10.8.125:42063,127.10.8.124:33929
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.8.66
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 01:43:22 UTC on e7f111948823
build id 13251
TSAN enabled
I20260809 01:45:21.976619 10994 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:45:21.978840 10994 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 01:45:21.992908 10993 heartbeater.cc:499] Master 127.10.8.126:42157 was elected leader, sending a full tablet report...
W20260809 01:45:21.995139 11017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:45:22.319146 11025 raft_consensus.cc:493] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 09a73ee8e8ec496a984efcab27715506)
I20260809 01:45:22.319916 11025 raft_consensus.cc:515] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:22.322893 11025 leader_election.cc:290] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063)
I20260809 01:45:22.325096 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:22.327083 10480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "71bc72deb1d2402cabd817dfe3b27e88" is_pre_election: true
I20260809 01:45:22.327826 10480 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 629f002048f64491afdd57f72a466e59 in term 1.
I20260809 01:45:22.329823 10581 leader_election.cc:304] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629f002048f64491afdd57f72a466e59, 71bc72deb1d2402cabd817dfe3b27e88; no voters:
I20260809 01:45:22.330874 11025 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 01:45:22.331717 11025 raft_consensus.cc:493] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 09a73ee8e8ec496a984efcab27715506)
I20260809 01:45:22.332824 11025 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 1 FOLLOWER]: Advancing to term 2
I20260809 01:45:22.333132 11027 raft_consensus.cc:493] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 09a73ee8e8ec496a984efcab27715506)
I20260809 01:45:22.333851 11027 raft_consensus.cc:515] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:22.337268 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:22.341192 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:45:22.342226 11027 leader_election.cc:290] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:22.343633 11025 raft_consensus.cc:515] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:22.346048 10612 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71bc72deb1d2402cabd817dfe3b27e88 in current term 2: Already voted for candidate 629f002048f64491afdd57f72a466e59 in this term.
I20260809 01:45:22.346630 11025 leader_election.cc:290] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 election: Requested vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063)
I20260809 01:45:22.349402 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506"
I20260809 01:45:22.350510 10480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629f002048f64491afdd57f72a466e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "71bc72deb1d2402cabd817dfe3b27e88"
I20260809 01:45:22.351671 10480 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 1 FOLLOWER]: Advancing to term 2
I20260809 01:45:22.353272 10441 leader_election.cc:304] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bc72deb1d2402cabd817dfe3b27e88; no voters: 09a73ee8e8ec496a984efcab27715506, 629f002048f64491afdd57f72a466e59
I20260809 01:45:22.362193 10480 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629f002048f64491afdd57f72a466e59 in term 2.
I20260809 01:45:22.362933 11027 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:45:22.363915 10581 leader_election.cc:304] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 629f002048f64491afdd57f72a466e59, 71bc72deb1d2402cabd817dfe3b27e88; no voters: 09a73ee8e8ec496a984efcab27715506
I20260809 01:45:22.365727 11025 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 01:45:22.368541 11025 raft_consensus.cc:697] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 2 LEADER]: Becoming Leader. State: Replica: 629f002048f64491afdd57f72a466e59, State: Running, Role: LEADER
I20260809 01:45:22.369899 11025 consensus_queue.cc:237] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:22.375890 11028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 629f002048f64491afdd57f72a466e59. Latest consensus state: current_term: 2 leader_uuid: "629f002048f64491afdd57f72a466e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:22.376780 11028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: This master's current role is: LEADER
I20260809 01:45:22.386853 11034 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 01:45:22.399293 11034 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 01:45:22.399672 10612 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [term 2 LEADER]: Rejecting Update request from peer 09a73ee8e8ec496a984efcab27715506 for earlier term 1. Current term is 2. Ops: []
I20260809 01:45:22.401999 11032 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 01:45:22.405380 11034 catalog_manager.cc:1295] Loaded cluster ID: 465fa00c323a404c8f72f4f6eb53200b
I20260809 01:45:22.405838 11034 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 01:45:22.410482 11032 consensus_queue.cc:1243] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [LEADER]: Peer 629f002048f64491afdd57f72a466e59 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260809 01:45:22.412189 11034 catalog_manager.cc:1540] Loading token signing keys...
I20260809 01:45:22.414446 11036 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 01:45:22.417734 11036 raft_consensus.cc:740] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 09a73ee8e8ec496a984efcab27715506, State: Running, Role: LEADER
I20260809 01:45:22.418403 11034 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Loaded TSK: 0
I20260809 01:45:22.419056 11036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:22.420367 11034 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 01:45:22.420830 11036 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 1 FOLLOWER]: Advancing to term 2
I20260809 01:45:22.568444 10744 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 01:45:22.842600 10362 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Refusing update from remote peer 629f002048f64491afdd57f72a466e59: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 01:45:22.845192 11028 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 01:45:22.854722 11036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 629f002048f64491afdd57f72a466e59. Latest consensus state: current_term: 2 leader_uuid: "629f002048f64491afdd57f72a466e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:22.856240 11036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:45:22.879387 10480 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Refusing update from remote peer 629f002048f64491afdd57f72a466e59: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 01:45:22.880561 11036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "629f002048f64491afdd57f72a466e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:22.882022 11036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:45:22.883562 11028 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 01:45:22.886387 11025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "629f002048f64491afdd57f72a466e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:22.887652 11025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: This master's current role is: LEADER
I20260809 01:45:22.910275 11043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 629f002048f64491afdd57f72a466e59. Latest consensus state: current_term: 2 leader_uuid: "629f002048f64491afdd57f72a466e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:22.911643 11043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:45:22.913188 11052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "629f002048f64491afdd57f72a466e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:22.914644 11052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:45:22.975159 10984 heartbeater.cc:499] Master 127.10.8.124:33929 was elected leader, sending a full tablet report...
I20260809 01:45:23.386550 11028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "629f002048f64491afdd57f72a466e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:23.387787 11028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: This master's current role is: LEADER
W20260809 01:45:23.398382 11015 debug-util.cc:398] Leaking SignalData structure 0x7b0800037500 after lost signal to thread 10994
W20260809 01:45:23.442807 11015 kernel_stack_watchdog.cc:198] Thread 10994 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 398ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260809 01:45:21.996379 11018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:45:23.444360 10994 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.447s user 0.000s sys 0.003s
W20260809 01:45:23.444803 10994 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.447s user 0.000s sys 0.003s
W20260809 01:45:23.450853 11022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:45:23.452522 10994 server_base.cc:1061] running on GCE node
I20260809 01:45:23.453810 10994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:45:23.456105 10994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:45:23.457479 10994 hybrid_clock.cc:648] HybridClock initialized: now 1786239923457437 us; error 49 us; skew 500 ppm
I20260809 01:45:23.458344 10994 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:45:23.464745 10994 webserver.cc:533] Webserver started at http://127.10.8.66:45909/ using document root <none> and password file <none>
I20260809 01:45:23.465783 10994 fs_manager.cc:362] Metadata directory not provided
I20260809 01:45:23.466039 10994 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:45:23.466534 10994 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:45:23.471170 10994 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/data/instance:
uuid: "33b5652538e64ed58a09993d3188f8d7"
format_stamp: "Formatted at 2026-08-09 01:45:23 on dist-test-slave-zr1t"
I20260809 01:45:23.472361 10994 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/instance:
uuid: "33b5652538e64ed58a09993d3188f8d7"
format_stamp: "Formatted at 2026-08-09 01:45:23 on dist-test-slave-zr1t"
I20260809 01:45:23.480965 10994 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260809 01:45:23.487936 11060 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:45:23.489276 10994 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.003s sys 0.003s
I20260809 01:45:23.489637 10994 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/data,/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal
uuid: "33b5652538e64ed58a09993d3188f8d7"
format_stamp: "Formatted at 2026-08-09 01:45:23 on dist-test-slave-zr1t"
I20260809 01:45:23.489979 10994 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:45:23.560499 10994 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 01:45:23.563318 10994 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:45:23.564005 10994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:45:23.569597 10994 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 01:45:23.577327 10994 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:45:23.577585 10994 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 01:45:23.577849 10994 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 01:45:23.577999 10994 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:45:23.889706 11162 raft_consensus.cc:493] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 629f002048f64491afdd57f72a466e59)
I20260809 01:45:23.890419 11162 raft_consensus.cc:515] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:23.892197 11162 leader_election.cc:290] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:23.893476 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:45:23.894200 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:23.895666 10439 leader_election.cc:304] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bc72deb1d2402cabd817dfe3b27e88; no voters: 09a73ee8e8ec496a984efcab27715506, 629f002048f64491afdd57f72a466e59
I20260809 01:45:23.896277 11162 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 01:45:24.054836 10994 rpc_server.cc:307] RPC server started. Bound to: 127.10.8.66:42495
I20260809 01:45:24.054961 11235 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.8.66:42495 every 8 connection(s)
I20260809 01:45:24.057617 10994 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/data/info.pb
I20260809 01:45:24.059671 10273 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu as pid 10994
I20260809 01:45:24.060173 10273 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/instance
I20260809 01:45:24.113596 10273 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
/tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.8.67:0
--local_ip_for_outbound_sockets=127.10.8.67
--webserver_interface=127.10.8.67
--webserver_port=0
--tserver_master_addrs=127.10.8.126:42157,127.10.8.125:42063,127.10.8.124:33929
--builtin_ntp_servers=127.10.8.84:40957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260809 01:45:24.114530 11239 heartbeater.cc:344] Connected to a master server at 127.10.8.125:42063
I20260809 01:45:24.115260 11239 heartbeater.cc:461] Registering TS with master...
I20260809 01:45:24.117014 11239 heartbeater.cc:507] Master 127.10.8.125:42063 requested a full tablet report, sending...
I20260809 01:45:24.122063 10464 ts_manager.cc:194] Registered new tserver with Master: 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:24.134130 11237 heartbeater.cc:344] Connected to a master server at 127.10.8.124:33929
I20260809 01:45:24.135579 11237 heartbeater.cc:461] Registering TS with master...
I20260809 01:45:24.136459 11237 heartbeater.cc:507] Master 127.10.8.124:33929 requested a full tablet report, sending...
I20260809 01:45:24.138792 11240 heartbeater.cc:344] Connected to a master server at 127.10.8.126:42157
I20260809 01:45:24.139154 11240 heartbeater.cc:461] Registering TS with master...
I20260809 01:45:24.139078 10599 ts_manager.cc:194] Registered new tserver with Master: 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:24.140086 11240 heartbeater.cc:507] Master 127.10.8.126:42157 requested a full tablet report, sending...
I20260809 01:45:24.142167 10599 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.10.8.66:57751
I20260809 01:45:24.142925 10344 ts_manager.cc:194] Registered new tserver with Master: 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
W20260809 01:45:24.574899 11251 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:45:24.575459 11251 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:45:24.576155 11251 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 01:45:24.611624 11251 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:45:24.612619 11251 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.8.67
I20260809 01:45:24.646420 11237 heartbeater.cc:499] Master 127.10.8.124:33929 was elected leader, sending a full tablet report...
I20260809 01:45:24.654939 11251 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.8.84:40957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.8.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.8.67
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.8.126:42157,127.10.8.125:42063,127.10.8.124:33929
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.8.67
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 01:43:22 UTC on e7f111948823
build id 13251
TSAN enabled
I20260809 01:45:24.656288 11251 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:45:24.658412 11251 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 01:45:24.675105 11259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:45:24.749195 11268 raft_consensus.cc:493] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 629f002048f64491afdd57f72a466e59)
I20260809 01:45:24.750087 11268 raft_consensus.cc:515] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:24.755626 10480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09a73ee8e8ec496a984efcab27715506" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "71bc72deb1d2402cabd817dfe3b27e88" is_pre_election: true
I20260809 01:45:24.760190 11268 leader_election.cc:290] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:24.761209 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09a73ee8e8ec496a984efcab27715506" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:45:24.765563 10319 leader_election.cc:304] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09a73ee8e8ec496a984efcab27715506; no voters: 629f002048f64491afdd57f72a466e59, 71bc72deb1d2402cabd817dfe3b27e88
I20260809 01:45:24.766790 11268 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 01:45:25.020843 11269 raft_consensus.cc:493] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 629f002048f64491afdd57f72a466e59)
I20260809 01:45:25.024209 11269 raft_consensus.cc:515] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:25.040911 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:25.047305 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:45:25.050285 10441 leader_election.cc:304] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bc72deb1d2402cabd817dfe3b27e88; no voters: 09a73ee8e8ec496a984efcab27715506, 629f002048f64491afdd57f72a466e59
I20260809 01:45:25.063498 11269 leader_election.cc:290] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:25.067171 11269 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 01:45:25.614665 11273 raft_consensus.cc:493] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 629f002048f64491afdd57f72a466e59)
I20260809 01:45:25.615806 11273 raft_consensus.cc:515] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:25.620639 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:45:25.620800 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:25.623311 10441 leader_election.cc:304] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bc72deb1d2402cabd817dfe3b27e88; no voters: 09a73ee8e8ec496a984efcab27715506, 629f002048f64491afdd57f72a466e59
I20260809 01:45:25.628412 11273 leader_election.cc:290] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:25.629926 11273 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260809 01:45:26.088817 11258 debug-util.cc:398] Leaking SignalData structure 0x7b0800037500 after lost signal to thread 11251
W20260809 01:45:26.530016 11251 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.854s user 0.587s sys 1.149s
W20260809 01:45:24.676805 11260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:45:26.531242 11251 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.855s user 0.587s sys 1.149s
W20260809 01:45:26.533135 11263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:45:26.536342 11261 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1855 milliseconds
I20260809 01:45:26.536389 11251 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260809 01:45:26.538116 11251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:45:26.542228 11251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:45:26.543787 11251 hybrid_clock.cc:648] HybridClock initialized: now 1786239926543722 us; error 46 us; skew 500 ppm
I20260809 01:45:26.545004 11251 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:45:26.556730 11251 webserver.cc:533] Webserver started at http://127.10.8.67:41603/ using document root <none> and password file <none>
I20260809 01:45:26.557928 11251 fs_manager.cc:362] Metadata directory not provided
I20260809 01:45:26.558276 11251 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:45:26.558760 11251 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:45:26.565306 11251 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/data/instance:
uuid: "4f60c9ac9414403d9ea5a97b103c2316"
format_stamp: "Formatted at 2026-08-09 01:45:26 on dist-test-slave-zr1t"
I20260809 01:45:26.566578 11251 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/instance:
uuid: "4f60c9ac9414403d9ea5a97b103c2316"
format_stamp: "Formatted at 2026-08-09 01:45:26 on dist-test-slave-zr1t"
I20260809 01:45:26.576884 11251 fs_manager.cc:696] Time spent creating directory manager: real 0.010s user 0.001s sys 0.009s
I20260809 01:45:26.586562 11282 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:45:26.587952 11251 fs_manager.cc:730] Time spent opening block manager: real 0.007s user 0.006s sys 0.002s
I20260809 01:45:26.588358 11251 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/data,/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal
uuid: "4f60c9ac9414403d9ea5a97b103c2316"
format_stamp: "Formatted at 2026-08-09 01:45:26 on dist-test-slave-zr1t"
I20260809 01:45:26.588753 11251 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:45:26.675058 11251 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 01:45:26.677489 11251 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:45:26.678190 11251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:45:26.684660 11251 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 01:45:26.691474 11251 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:45:26.691705 11251 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:45:26.691970 11251 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 01:45:26.692152 11251 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:45:27.122848 11251 rpc_server.cc:307] RPC server started. Bound to: 127.10.8.67:40895
I20260809 01:45:27.122982 11462 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.8.67:40895 every 8 connection(s)
I20260809 01:45:27.125748 11251 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/data/info.pb
I20260809 01:45:27.129359 10273 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu as pid 11251
I20260809 01:45:27.129938 10273 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/instance
I20260809 01:45:27.226387 11463 heartbeater.cc:344] Connected to a master server at 127.10.8.124:33929
I20260809 01:45:27.227092 11465 heartbeater.cc:344] Connected to a master server at 127.10.8.125:42063
I20260809 01:45:27.227221 11463 heartbeater.cc:461] Registering TS with master...
I20260809 01:45:27.227402 11465 heartbeater.cc:461] Registering TS with master...
I20260809 01:45:27.228914 11463 heartbeater.cc:507] Master 127.10.8.124:33929 requested a full tablet report, sending...
I20260809 01:45:27.230126 11465 heartbeater.cc:507] Master 127.10.8.125:42063 requested a full tablet report, sending...
I20260809 01:45:27.230436 11466 heartbeater.cc:344] Connected to a master server at 127.10.8.126:42157
I20260809 01:45:27.230918 11466 heartbeater.cc:461] Registering TS with master...
I20260809 01:45:27.232889 11466 heartbeater.cc:507] Master 127.10.8.126:42157 requested a full tablet report, sending...
I20260809 01:45:27.235863 10599 ts_manager.cc:194] Registered new tserver with Master: 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:27.236675 10464 ts_manager.cc:194] Registered new tserver with Master: 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:27.238385 10341 ts_manager.cc:194] Registered new tserver with Master: 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:27.238622 10599 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.10.8.67:40463
I20260809 01:45:27.264987 10273 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 01:45:27.376088 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_21d3c468b4c641bbb4a9f1832b00c21e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:27.377859 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_d22d96ad579446608e2d6db5fc64d0ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:27.387516 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_f7bfc81054974aeeafc596f3b72b3e97"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:27.393106 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d22d96ad579446608e2d6db5fc64d0ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:27.394368 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_21d3c468b4c641bbb4a9f1832b00c21e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:27.395490 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7bfc81054974aeeafc596f3b72b3e97 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:27.397548 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_1255c1af765f47a1a701e36cf70e0dcc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:27.402891 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1255c1af765f47a1a701e36cf70e0dcc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:27.452414 10273 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu with pid 10404
I20260809 01:45:27.479501 10596 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_1255c1af765f47a1a701e36cf70e0dcc" } new_table_name: "default.Table_55f55a64499e4eef89aea24d104bd81c" modify_external_catalogs: true
I20260809 01:45:27.501493 10599 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_f7bfc81054974aeeafc596f3b72b3e97" } new_table_name: "default.Table_04dac09c53c84a28a2f14c19104f5c9d" modify_external_catalogs: true
I20260809 01:45:27.502401 10597 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_21d3c468b4c641bbb4a9f1832b00c21e" } modify_external_catalogs: true
I20260809 01:45:27.503063 10597 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_21d3c468b4c641bbb4a9f1832b00c21e" } modify_external_catalogs: true
I20260809 01:45:27.508348 10598 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_d22d96ad579446608e2d6db5fc64d0ae" } modify_external_catalogs: true
I20260809 01:45:27.508898 10598 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_d22d96ad579446608e2d6db5fc64d0ae" } modify_external_catalogs: true
W20260809 01:45:27.512410 10581 connection.cc:573] client connection to 127.10.8.125:42063 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:45:27.512885 10581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:45:27.527925 10596 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9bdd0ebb4ff348db986bf29913d430ff consensus state has no leader
I20260809 01:45:27.553481 10598 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 152aba442eaa40f29c9cb66f5ebf08bf
W20260809 01:45:27.561736 10986 heartbeater.cc:646] Failed to heartbeat to 127.10.8.125:42063 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
W20260809 01:45:27.566972 10581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.125:42063, user_credentials={real_user=slave}} blocked reactor thread for 54119.2us
W20260809 01:45:27.577361 10599 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 949507eb878d470c8cf67abe0291f8eb has no consensus state
I20260809 01:45:27.590456 10597 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 8ce9c23b969a42c1a9a30868679b4033
I20260809 01:45:27.597110 11129 tablet_service.cc:1511] Processing CreateTablet for tablet 9bdd0ebb4ff348db986bf29913d430ff (DEFAULT_TABLE table=default.Table_1255c1af765f47a1a701e36cf70e0dcc [id=ff18a4c4305e465caf0a5801ccbee7a1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:27.604988 10581 consensus_peers.cc:597] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 -> Peer 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063): Couldn't send request to peer 71bc72deb1d2402cabd817dfe3b27e88. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:27.605008 11369 tablet_service.cc:1511] Processing CreateTablet for tablet 9bdd0ebb4ff348db986bf29913d430ff (DEFAULT_TABLE table=default.Table_55f55a64499e4eef89aea24d104bd81c [id=ff18a4c4305e465caf0a5801ccbee7a1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:27.606088 10581 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 9bdd0ebb4ff348db986bf29913d430ff (table default.Table_55f55a64499e4eef89aea24d104bd81c [id=ff18a4c4305e465caf0a5801ccbee7a1], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9bdd0ebb4ff348db986bf29913d430ff consensus state has no leader
I20260809 01:45:27.606932 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bdd0ebb4ff348db986bf29913d430ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:27.607105 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bdd0ebb4ff348db986bf29913d430ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:27.610533 10877 tablet_service.cc:1511] Processing CreateTablet for tablet 9bdd0ebb4ff348db986bf29913d430ff (DEFAULT_TABLE table=default.Table_55f55a64499e4eef89aea24d104bd81c [id=ff18a4c4305e465caf0a5801ccbee7a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:27.612941 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bdd0ebb4ff348db986bf29913d430ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:27.622671 10874 tablet_service.cc:1511] Processing CreateTablet for tablet 949507eb878d470c8cf67abe0291f8eb (DEFAULT_TABLE table=default.Table_04dac09c53c84a28a2f14c19104f5c9d [id=6a7b6884b2e74006a43ce56bc5c264ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:27.623921 10874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 949507eb878d470c8cf67abe0291f8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:27.624576 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 949507eb878d470c8cf67abe0291f8eb (DEFAULT_TABLE table=default.Table_04dac09c53c84a28a2f14c19104f5c9d [id=6a7b6884b2e74006a43ce56bc5c264ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:27.626101 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 949507eb878d470c8cf67abe0291f8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:27.625846 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 949507eb878d470c8cf67abe0291f8eb (DEFAULT_TABLE table=default.Table_04dac09c53c84a28a2f14c19104f5c9d [id=6a7b6884b2e74006a43ce56bc5c264ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:27.627142 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 949507eb878d470c8cf67abe0291f8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:27.667449 11551 tablet_bootstrap.cc:492] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:27.671146 11553 tablet_bootstrap.cc:492] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:27.681491 11551 tablet_bootstrap.cc:654] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:27.682375 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_cb5a243ff9654c1f8a4a8bc6736e626c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:27.684094 11551 log.cc:826] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7: Log is configured to *not* fsync() on all Append() calls
W20260809 01:45:27.684250 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cb5a243ff9654c1f8a4a8bc6736e626c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:27.690430 11239 heartbeater.cc:646] Failed to heartbeat to 127.10.8.125:42063 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
I20260809 01:45:27.692157 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_aed6870dac374d5d9ecc995f90b09201"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:27.693109 11553 tablet_bootstrap.cc:654] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:27.695698 11553 log.cc:826] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8: Log is configured to *not* fsync() on all Append() calls
I20260809 01:45:27.697992 11554 tablet_bootstrap.cc:492] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
W20260809 01:45:27.712184 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aed6870dac374d5d9ecc995f90b09201 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:27.713603 11554 tablet_bootstrap.cc:654] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:27.717629 11554 log.cc:826] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316: Log is configured to *not* fsync() on all Append() calls
I20260809 01:45:27.718601 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_aaad8afaf8134c808577f0b370975c6c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:27.720504 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aaad8afaf8134c808577f0b370975c6c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:27.724466 11553 tablet_bootstrap.cc:492] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:27.725082 11553 ts_tablet_manager.cc:1403] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.054s user 0.008s sys 0.014s
I20260809 01:45:27.722919 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_844ab3758b7b424ead554a23317aa4bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:27.730173 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_844ab3758b7b424ead554a23317aa4bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:27.732450 11554 tablet_bootstrap.cc:492] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:27.733172 11554 ts_tablet_manager.cc:1403] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.036s user 0.020s sys 0.008s
I20260809 01:45:27.740664 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_55f55a64499e4eef89aea24d104bd81c" } new_table_name: "default.Table_ecca25c58193453e8cf32461e3af95a9" modify_external_catalogs: true
I20260809 01:45:27.743101 11463 heartbeater.cc:499] Master 127.10.8.124:33929 was elected leader, sending a full tablet report...
I20260809 01:45:27.771392 10592 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_04dac09c53c84a28a2f14c19104f5c9d" } modify_external_catalogs: true
I20260809 01:45:27.772046 10592 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_04dac09c53c84a28a2f14c19104f5c9d" } modify_external_catalogs: true
I20260809 01:45:27.776456 11554 raft_consensus.cc:359] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:27.777979 11554 raft_consensus.cc:385] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:27.778518 11554 raft_consensus.cc:740] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:27.779943 11554 consensus_queue.cc:260] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:27.783960 10874 tablet_service.cc:1511] Processing CreateTablet for tablet f3c0c5eaa1f64c0298226901de71e7f9 (DEFAULT_TABLE table=default.Table_cb5a243ff9654c1f8a4a8bc6736e626c [id=71f0e1b66e0d466ba1297d4ce6c85ad9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:27.785706 10874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3c0c5eaa1f64c0298226901de71e7f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:27.789743 11127 tablet_service.cc:1511] Processing CreateTablet for tablet f3c0c5eaa1f64c0298226901de71e7f9 (DEFAULT_TABLE table=default.Table_cb5a243ff9654c1f8a4a8bc6736e626c [id=71f0e1b66e0d466ba1297d4ce6c85ad9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:27.791010 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3c0c5eaa1f64c0298226901de71e7f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:27.793331 11551 tablet_bootstrap.cc:492] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:27.793907 11551 ts_tablet_manager.cc:1403] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.127s user 0.003s sys 0.020s
I20260809 01:45:27.804303 10591 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_aed6870dac374d5d9ecc995f90b09201" } modify_external_catalogs: true
I20260809 01:45:27.804880 10591 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_aed6870dac374d5d9ecc995f90b09201" } modify_external_catalogs: true
I20260809 01:45:27.806303 10599 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_cb5a243ff9654c1f8a4a8bc6736e626c" } new_table_name: "default.Table_f497f9f1fa2f4ca3833bef3bae66605e" modify_external_catalogs: true
I20260809 01:45:27.807834 11554 ts_tablet_manager.cc:1434] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.074s user 0.044s sys 0.004s
I20260809 01:45:27.809345 11554 tablet_bootstrap.cc:492] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:27.837075 11554 tablet_bootstrap.cc:654] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:27.857725 10599 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f3c0c5eaa1f64c0298226901de71e7f9 consensus state has no leader
I20260809 01:45:27.864071 10874 tablet_service.cc:1511] Processing CreateTablet for tablet 5b2b6684972a477e83b19d977f0c1e7c (DEFAULT_TABLE table=default.Table_aed6870dac374d5d9ecc995f90b09201 [id=1fc93322377e409b961a543dcd0d56f5]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:27.865307 11465 heartbeater.cc:646] Failed to heartbeat to 127.10.8.125:42063 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
I20260809 01:45:27.865768 10874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b2b6684972a477e83b19d977f0c1e7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:27.867903 11369 tablet_service.cc:1511] Processing CreateTablet for tablet f3c0c5eaa1f64c0298226901de71e7f9 (DEFAULT_TABLE table=default.Table_cb5a243ff9654c1f8a4a8bc6736e626c [id=71f0e1b66e0d466ba1297d4ce6c85ad9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:27.870165 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3c0c5eaa1f64c0298226901de71e7f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:27.870280 11129 tablet_service.cc:1511] Processing CreateTablet for tablet 5b2b6684972a477e83b19d977f0c1e7c (DEFAULT_TABLE table=default.Table_aed6870dac374d5d9ecc995f90b09201 [id=1fc93322377e409b961a543dcd0d56f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:27.872157 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b2b6684972a477e83b19d977f0c1e7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:27.894230 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 5b2b6684972a477e83b19d977f0c1e7c (DEFAULT_TABLE table=default.Table_aed6870dac374d5d9ecc995f90b09201 [id=1fc93322377e409b961a543dcd0d56f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:27.898250 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b2b6684972a477e83b19d977f0c1e7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:27.901929 10591 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 5b2b6684972a477e83b19d977f0c1e7c
I20260809 01:45:27.906000 10877 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b2b6684972a477e83b19d977f0c1e7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:27 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:27.906430 11365 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b2b6684972a477e83b19d977f0c1e7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:27 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:27.911074 11126 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b2b6684972a477e83b19d977f0c1e7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:27 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:27.924664 11554 tablet_bootstrap.cc:492] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:27.926079 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_006a6ebb9836470396c69b4b78abc954"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:27.927553 11554 ts_tablet_manager.cc:1403] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.119s user 0.025s sys 0.009s
W20260809 01:45:27.928804 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_006a6ebb9836470396c69b4b78abc954 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:27.930948 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 5b2b6684972a477e83b19d977f0c1e7c because the tablet was not found. No further retry: Not found: Tablet not found: 5b2b6684972a477e83b19d977f0c1e7c
W20260809 01:45:27.935685 11470 tablet.cc:2406] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 01:45:27.936228 10576 catalog_manager.cc:5017] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 5b2b6684972a477e83b19d977f0c1e7c because the tablet was not found. No further retry: Not found: Tablet not found: 5b2b6684972a477e83b19d977f0c1e7c
I20260809 01:45:27.936708 10595 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 9bdd0ebb4ff348db986bf29913d430ff (table default.Table_55f55a64499e4eef89aea24d104bd81c [id=ff18a4c4305e465caf0a5801ccbee7a1]). Expected version 1 got 0
I20260809 01:45:27.937870 10592 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 949507eb878d470c8cf67abe0291f8eb
I20260809 01:45:27.941143 11365 tablet_service.cc:1558] Processing DeleteTablet for tablet 949507eb878d470c8cf67abe0291f8eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:27 UTC) from {username='slave'} at 127.0.0.1:53476
W20260809 01:45:27.942764 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9bdd0ebb4ff348db986bf29913d430ff consensus state has no leader
W20260809 01:45:27.944279 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 949507eb878d470c8cf67abe0291f8eb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 949507eb878d470c8cf67abe0291f8eb already in progress: creating tablet
I20260809 01:45:27.950846 11554 raft_consensus.cc:359] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:27.954553 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet 949507eb878d470c8cf67abe0291f8eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:27 UTC) from {username='slave'} at 127.0.0.1:45982
W20260809 01:45:27.960604 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 5b2b6684972a477e83b19d977f0c1e7c because the tablet was not found. No further retry: Not found: Tablet not found: 5b2b6684972a477e83b19d977f0c1e7c
W20260809 01:45:27.963537 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 949507eb878d470c8cf67abe0291f8eb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 949507eb878d470c8cf67abe0291f8eb already in progress: creating tablet
I20260809 01:45:27.967175 11523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "629f002048f64491afdd57f72a466e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:27.970098 11523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: This master's current role is: LEADER
I20260809 01:45:27.971176 11551 raft_consensus.cc:359] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:27.972484 11551 raft_consensus.cc:385] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:27.972849 11551 raft_consensus.cc:740] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:27.973906 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_995c53816d9745ec9293d59a7cee814f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:27.973999 11551 consensus_queue.cc:260] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:27.975793 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_995c53816d9745ec9293d59a7cee814f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:27.980904 11129 tablet_service.cc:1558] Processing DeleteTablet for tablet 949507eb878d470c8cf67abe0291f8eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:27 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:27.996246 11551 ts_tablet_manager.cc:1434] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.170s user 0.031s sys 0.011s
I20260809 01:45:28.013804 10597 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 9bdd0ebb4ff348db986bf29913d430ff (table default.Table_ecca25c58193453e8cf32461e3af95a9 [id=ff18a4c4305e465caf0a5801ccbee7a1]). Expected version 2 got 0
I20260809 01:45:28.018195 11551 tablet_bootstrap.cc:492] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:28.022766 10877 tablet_service.cc:1511] Processing CreateTablet for tablet dfb4ad36ca954c7c99407612c0f30711 (DEFAULT_TABLE table=default.Table_844ab3758b7b424ead554a23317aa4bc [id=7831718cddee4b5b9c42cec19ff7d074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.024155 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfb4ad36ca954c7c99407612c0f30711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.024943 11551 tablet_bootstrap.cc:654] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:28.025826 11129 tablet_service.cc:1511] Processing CreateTablet for tablet dfb4ad36ca954c7c99407612c0f30711 (DEFAULT_TABLE table=default.Table_844ab3758b7b424ead554a23317aa4bc [id=7831718cddee4b5b9c42cec19ff7d074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.027465 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfb4ad36ca954c7c99407612c0f30711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.027945 11565 tablet_replica.cc:333] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:28.029259 11565 raft_consensus.cc:2243] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:28.030526 11565 raft_consensus.cc:2272] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:28.033891 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_421f0008df9b4556b82b41e0c67d277a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:28.036211 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_421f0008df9b4556b82b41e0c67d277a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:28.040619 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_13597d7e311b4ff59dc3782447360563"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:28.042321 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13597d7e311b4ff59dc3782447360563 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:27.958293 11365 tablet_service.cc:1558] Processing DeleteTablet for tablet 949507eb878d470c8cf67abe0291f8eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:27 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:28.046448 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 64ac88034c7a4dbb82a39ac7731c37e5 (DEFAULT_TABLE table=default.Table_aaad8afaf8134c808577f0b370975c6c [id=7ff740ce8d4f4b5995b4d5ef52cafada]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.023847 11364 tablet_service.cc:1511] Processing CreateTablet for tablet dfb4ad36ca954c7c99407612c0f30711 (DEFAULT_TABLE table=default.Table_844ab3758b7b424ead554a23317aa4bc [id=7831718cddee4b5b9c42cec19ff7d074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.047591 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ac88034c7a4dbb82a39ac7731c37e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.048255 11364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfb4ad36ca954c7c99407612c0f30711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.026670 11560 raft_consensus.cc:493] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:28.049979 11560 raft_consensus.cc:515] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:28.054956 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 949507eb878d470c8cf67abe0291f8eb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 949507eb878d470c8cf67abe0291f8eb already in progress: creating tablet
I20260809 01:45:28.057663 11365 tablet_service.cc:1511] Processing CreateTablet for tablet 64ac88034c7a4dbb82a39ac7731c37e5 (DEFAULT_TABLE table=default.Table_aaad8afaf8134c808577f0b370975c6c [id=7ff740ce8d4f4b5995b4d5ef52cafada]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.059383 11365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ac88034c7a4dbb82a39ac7731c37e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.062071 10599 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_f497f9f1fa2f4ca3833bef3bae66605e" } new_table_name: "default.Table_6149470a609f459c951f42acf3332a0d" modify_external_catalogs: true
I20260809 01:45:28.102937 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bdd0ebb4ff348db986bf29913d430ff" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:28.106191 11283 leader_election.cc:343] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:28.107527 11560 leader_election.cc:290] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:28.128499 11551 tablet_bootstrap.cc:492] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:28.128988 11551 ts_tablet_manager.cc:1403] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.111s user 0.020s sys 0.010s
I20260809 01:45:28.131891 11551 raft_consensus.cc:359] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:28.132507 11551 raft_consensus.cc:385] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:28.132763 11551 raft_consensus.cc:740] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:28.133111 11554 raft_consensus.cc:385] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:28.133402 11551 consensus_queue.cc:260] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:28.139045 10592 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_995c53816d9745ec9293d59a7cee814f" } modify_external_catalogs: true
I20260809 01:45:28.139600 10592 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_995c53816d9745ec9293d59a7cee814f" } modify_external_catalogs: true
I20260809 01:45:28.141673 11554 raft_consensus.cc:740] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:28.143590 11554 consensus_queue.cc:260] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:28.159586 11551 ts_tablet_manager.cc:1434] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260809 01:45:28.160631 11551 tablet_bootstrap.cc:492] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:28.166884 11551 tablet_bootstrap.cc:654] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:28.167964 10597 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_aaad8afaf8134c808577f0b370975c6c" } modify_external_catalogs: true
I20260809 01:45:28.168545 10597 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_aaad8afaf8134c808577f0b370975c6c" } modify_external_catalogs: true
W20260809 01:45:28.172938 10599 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f3c0c5eaa1f64c0298226901de71e7f9 consensus state has no leader
I20260809 01:45:28.174573 10595 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_844ab3758b7b424ead554a23317aa4bc" } new_table_name: "default.Table_d643f50faae24194b5f791d79b9905d3" modify_external_catalogs: true
I20260809 01:45:28.175463 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet 949507eb878d470c8cf67abe0291f8eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:27 UTC) from {username='slave'} at 127.0.0.1:53476
W20260809 01:45:28.177066 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 949507eb878d470c8cf67abe0291f8eb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 949507eb878d470c8cf67abe0291f8eb already in progress: creating tablet
I20260809 01:45:28.191857 10592 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 6a7693be7a6543b588c3ea6bfb0461c3
I20260809 01:45:28.195362 11554 ts_tablet_manager.cc:1434] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.267s user 0.004s sys 0.004s
I20260809 01:45:28.199644 10591 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 9bdd0ebb4ff348db986bf29913d430ff (table default.Table_ecca25c58193453e8cf32461e3af95a9 [id=ff18a4c4305e465caf0a5801ccbee7a1]). Expected version 2 got 0
I20260809 01:45:28.034602 11565 ts_tablet_manager.cc:1916] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:28.208073 10599 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 9bdd0ebb4ff348db986bf29913d430ff (table default.Table_ecca25c58193453e8cf32461e3af95a9 [id=ff18a4c4305e465caf0a5801ccbee7a1]). Expected version 2 got 0
I20260809 01:45:28.209097 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_2a4ece19682a41cea3b9b7737f78304f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:28.214524 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2a4ece19682a41cea3b9b7737f78304f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:28.214898 11551 tablet_bootstrap.cc:492] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:28.215322 11551 ts_tablet_manager.cc:1403] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.055s user 0.007s sys 0.005s
I20260809 01:45:28.218043 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet 949507eb878d470c8cf67abe0291f8eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:27 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:28.218676 11551 raft_consensus.cc:359] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:28.219643 11551 raft_consensus.cc:385] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:28.220067 11551 raft_consensus.cc:740] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:28.220785 11551 consensus_queue.cc:260] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:28.222882 11551 ts_tablet_manager.cc:1434] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 01:45:28.223773 11551 tablet_bootstrap.cc:492] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:28.234436 10599 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet f3c0c5eaa1f64c0298226901de71e7f9 (table default.Table_6149470a609f459c951f42acf3332a0d [id=71f0e1b66e0d466ba1297d4ce6c85ad9]). Expected version 2 got 0
I20260809 01:45:28.246017 11551 tablet_bootstrap.cc:654] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:28.236776 10873 tablet_service.cc:1511] Processing CreateTablet for tablet 64ac88034c7a4dbb82a39ac7731c37e5 (DEFAULT_TABLE table=default.Table_aaad8afaf8134c808577f0b370975c6c [id=7ff740ce8d4f4b5995b4d5ef52cafada]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.249213 11564 tablet_replica.cc:333] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:28.250525 11564 raft_consensus.cc:2243] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:28.251376 11564 raft_consensus.cc:2272] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:28.256472 11564 ts_tablet_manager.cc:1916] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:28.246416 10595 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dfb4ad36ca954c7c99407612c0f30711 consensus state has no leader
I20260809 01:45:28.256198 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 64ac88034c7a4dbb82a39ac7731c37e5
I20260809 01:45:28.272411 11129 tablet_service.cc:1558] Processing DeleteTablet for tablet 64ac88034c7a4dbb82a39ac7731c37e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:28.272187 10871 tablet_service.cc:1511] Processing CreateTablet for tablet 825c568f98014fbf92ade7bd99763581 (DEFAULT_TABLE table=default.Table_006a6ebb9836470396c69b4b78abc954 [id=366d143256e84961b8b8ae2bc08452d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.274179 11129 tablet_service.cc:1511] Processing CreateTablet for tablet 825c568f98014fbf92ade7bd99763581 (DEFAULT_TABLE table=default.Table_006a6ebb9836470396c69b4b78abc954 [id=366d143256e84961b8b8ae2bc08452d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.274559 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 64ac88034c7a4dbb82a39ac7731c37e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:28.275260 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 825c568f98014fbf92ade7bd99763581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.277112 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ac88034c7a4dbb82a39ac7731c37e5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:28.278312 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 64ac88034c7a4dbb82a39ac7731c37e5 because the tablet was not found. No further retry: Not found: Tablet not found: 64ac88034c7a4dbb82a39ac7731c37e5
I20260809 01:45:28.296857 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 825c568f98014fbf92ade7bd99763581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.317008 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_d37c43f0673044eb826f3944101bcd4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:28.318684 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d37c43f0673044eb826f3944101bcd4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:28.324116 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_1cfd8f5e076b4fb49235142ceb2e9bae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:28.325755 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1cfd8f5e076b4fb49235142ceb2e9bae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:28.327886 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet 64ac88034c7a4dbb82a39ac7731c37e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:28.331115 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 825c568f98014fbf92ade7bd99763581 (DEFAULT_TABLE table=default.Table_006a6ebb9836470396c69b4b78abc954 [id=366d143256e84961b8b8ae2bc08452d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.353251 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 825c568f98014fbf92ade7bd99763581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.362375 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_2b6ecadf1f6741229c08338a74153bf8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:28.364104 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b6ecadf1f6741229c08338a74153bf8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:28.386538 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_2a4ece19682a41cea3b9b7737f78304f" } new_table_name: "default.Table_0153cf456ad34c5f9d96f03130ffb5bb" modify_external_catalogs: true
I20260809 01:45:28.386333 11553 raft_consensus.cc:359] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:28.388062 11553 raft_consensus.cc:385] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:28.388594 11553 raft_consensus.cc:740] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
W20260809 01:45:28.409155 11246 tablet.cc:2406] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:45:28.420459 10599 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_1cfd8f5e076b4fb49235142ceb2e9bae" } modify_external_catalogs: true
I20260809 01:45:28.421209 10599 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_1cfd8f5e076b4fb49235142ceb2e9bae" } modify_external_catalogs: true
I20260809 01:45:28.431257 11129 tablet_service.cc:1511] Processing CreateTablet for tablet a8c4892ed69143cfb9659a6f4a9d9e55 (DEFAULT_TABLE table=default.Table_13597d7e311b4ff59dc3782447360563 [id=dbdbdc9a4bd54ba2811644b79f56a451]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.432467 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8c4892ed69143cfb9659a6f4a9d9e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.433867 11553 consensus_queue.cc:260] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:28.438444 11366 tablet_service.cc:1511] Processing CreateTablet for tablet a8c4892ed69143cfb9659a6f4a9d9e55 (DEFAULT_TABLE table=default.Table_13597d7e311b4ff59dc3782447360563 [id=dbdbdc9a4bd54ba2811644b79f56a451]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.440193 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8c4892ed69143cfb9659a6f4a9d9e55. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:28.442986 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 72b4850de7d740d794283ca93152fdc1 has no consensus state
I20260809 01:45:28.451014 10870 tablet_service.cc:1511] Processing CreateTablet for tablet e155d16b1d594a789f539390a44bc39f (DEFAULT_TABLE table=default.Table_421f0008df9b4556b82b41e0c67d277a [id=966459ae200d45429a01326bc653f8d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.430387 10874 tablet_service.cc:1511] Processing CreateTablet for tablet a8c4892ed69143cfb9659a6f4a9d9e55 (DEFAULT_TABLE table=default.Table_13597d7e311b4ff59dc3782447360563 [id=dbdbdc9a4bd54ba2811644b79f56a451]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.455480 11127 tablet_service.cc:1511] Processing CreateTablet for tablet e155d16b1d594a789f539390a44bc39f (DEFAULT_TABLE table=default.Table_421f0008df9b4556b82b41e0c67d277a [id=966459ae200d45429a01326bc653f8d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.456635 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e155d16b1d594a789f539390a44bc39f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.458758 11551 tablet_bootstrap.cc:492] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:28.459474 11551 ts_tablet_manager.cc:1403] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.236s user 0.035s sys 0.041s
I20260809 01:45:28.463639 11570 raft_consensus.cc:493] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:28.464425 11570 raft_consensus.cc:515] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:28.473718 11570 leader_election.cc:290] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:28.476660 10599 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet a8b2804544e243e6ba22c87a743f6425
I20260809 01:45:28.480600 11551 raft_consensus.cc:359] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:28.486177 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e155d16b1d594a789f539390a44bc39f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.509495 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 48b7459ad0974cb9948d647459b22260 (DEFAULT_TABLE table=default.Table_d37c43f0673044eb826f3944101bcd4b [id=ec2be55379204f57a7bf8a8739b73033]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.511291 10874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8c4892ed69143cfb9659a6f4a9d9e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.512780 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48b7459ad0974cb9948d647459b22260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.517328 10867 tablet_service.cc:1511] Processing CreateTablet for tablet 72b4850de7d740d794283ca93152fdc1 (DEFAULT_TABLE table=default.Table_0153cf456ad34c5f9d96f03130ffb5bb [id=46b096a453084568935bc0d5855cb0f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.519037 10867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72b4850de7d740d794283ca93152fdc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.510073 11129 tablet_service.cc:1511] Processing CreateTablet for tablet 48b7459ad0974cb9948d647459b22260 (DEFAULT_TABLE table=default.Table_d37c43f0673044eb826f3944101bcd4b [id=ec2be55379204f57a7bf8a8739b73033]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.543231 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48b7459ad0974cb9948d647459b22260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.518126 11126 tablet_service.cc:1511] Processing CreateTablet for tablet 72b4850de7d740d794283ca93152fdc1 (DEFAULT_TABLE table=default.Table_0153cf456ad34c5f9d96f03130ffb5bb [id=46b096a453084568935bc0d5855cb0f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.546706 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72b4850de7d740d794283ca93152fdc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.555378 10591 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_0153cf456ad34c5f9d96f03130ffb5bb" } new_table_name: "default.Table_8f1bad3be6ed41f086fd8a9cf367555a" modify_external_catalogs: true
I20260809 01:45:28.555594 10599 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_ecca25c58193453e8cf32461e3af95a9" } modify_external_catalogs: true
I20260809 01:45:28.556063 10599 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_ecca25c58193453e8cf32461e3af95a9" } modify_external_catalogs: true
I20260809 01:45:28.482872 11369 tablet_service.cc:1511] Processing CreateTablet for tablet e155d16b1d594a789f539390a44bc39f (DEFAULT_TABLE table=default.Table_421f0008df9b4556b82b41e0c67d277a [id=966459ae200d45429a01326bc653f8d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.558692 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 72b4850de7d740d794283ca93152fdc1 (DEFAULT_TABLE table=default.Table_0153cf456ad34c5f9d96f03130ffb5bb [id=46b096a453084568935bc0d5855cb0f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.559628 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e155d16b1d594a789f539390a44bc39f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.560865 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_cad8aab384fb4dd59f29d69bc4798da7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:28.562614 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cad8aab384fb4dd59f29d69bc4798da7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:28.563421 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72b4850de7d740d794283ca93152fdc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.521925 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bdd0ebb4ff348db986bf29913d430ff" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:28.565922 11398 raft_consensus.cc:2468] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:28.568821 11062 leader_election.cc:304] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:28.569880 11570 raft_consensus.cc:2804] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:28.571053 11570 raft_consensus.cc:493] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:28.572119 11570 raft_consensus.cc:3060] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:28.556596 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 48b7459ad0974cb9948d647459b22260 (DEFAULT_TABLE table=default.Table_d37c43f0673044eb826f3944101bcd4b [id=ec2be55379204f57a7bf8a8739b73033]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.574316 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48b7459ad0974cb9948d647459b22260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.593681 11551 raft_consensus.cc:385] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:28.594321 11551 raft_consensus.cc:740] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:28.595649 11551 consensus_queue.cc:260] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:28.596854 10591 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 72b4850de7d740d794283ca93152fdc1 consensus state has no leader
I20260809 01:45:28.603432 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_14857c122d584ff994a4159a0fa99b98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:28.604826 11554 tablet_bootstrap.cc:492] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:28.611011 11554 tablet_bootstrap.cc:654] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:28.605392 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14857c122d584ff994a4159a0fa99b98 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:28.620436 11564 ts_tablet_manager.cc:1929] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:28.620924 11564 log.cc:1199] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/949507eb878d470c8cf67abe0291f8eb
I20260809 01:45:28.621857 11564 ts_tablet_manager.cc:1950] T 949507eb878d470c8cf67abe0291f8eb P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:28.621918 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bdd0ebb4ff348db986bf29913d430ff" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:28.652676 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_7f7d5bf058bd48ff8e3660ca567a5718"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:28.656765 11553 ts_tablet_manager.cc:1434] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.931s user 0.029s sys 0.017s
W20260809 01:45:28.657307 11553 ts_tablet_manager.cc:1467] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8: Tablet startup took 931ms
W20260809 01:45:28.658015 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f7d5bf058bd48ff8e3660ca567a5718 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:28.657871 11553 ts_tablet_manager.cc:1469] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8: Trace:
0809 01:45:27.621382 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:27.621617 (+ 235us) service_pool.cc:224] Handling call
0809 01:45:27.623431 (+ 1814us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:27.623515 (+ 84us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:27.639645 (+ 16130us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:27.648711 (+ 9066us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:27.670314 (+ 21603us) inbound_call.cc:177] Queueing success response
0809 01:45:27.670485 (+ 171us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0809 01:45:27.725446 (+ 54961us) ts_tablet_manager.cc:1435] Starting tablet replica
0809 01:45:27.726136 (+ 690us) tablet_replica.cc:261] Starting instrumentation
0809 01:45:27.726827 (+ 691us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":177264,"raft.run_cpu_time_us":529,"raft.run_wall_time_us":9894,"tablet-open.queue_time_us":20859,"thread_start_us":226408,"threads_started":2}
W20260809 01:45:28.691062 10579 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 72b4850de7d740d794283ca93152fdc1 (table default.Table_8f1bad3be6ed41f086fd8a9cf367555a [id=46b096a453084568935bc0d5855cb0f7], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 72b4850de7d740d794283ca93152fdc1 consensus state has no leader
W20260809 01:45:28.702875 11063 leader_election.cc:343] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:28.720398 11554 tablet_bootstrap.cc:492] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:28.720984 11554 ts_tablet_manager.cc:1403] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.116s user 0.015s sys 0.000s
I20260809 01:45:28.725950 11554 raft_consensus.cc:359] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:28.726925 11554 raft_consensus.cc:385] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:28.727357 11554 raft_consensus.cc:740] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:28.728771 11554 consensus_queue.cc:260] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:28.744365 10581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.125:42063, user_credentials={real_user=slave}} blocked reactor thread for 77285.7us
I20260809 01:45:28.753285 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_2b6ecadf1f6741229c08338a74153bf8" } new_table_name: "default.Table_b9f38ba904ef48559bddb776d4f3a4ff" modify_external_catalogs: true
I20260809 01:45:28.757455 10595 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_13597d7e311b4ff59dc3782447360563" } modify_external_catalogs: true
I20260809 01:45:28.762498 10595 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_13597d7e311b4ff59dc3782447360563" } modify_external_catalogs: true
I20260809 01:45:28.780894 11554 ts_tablet_manager.cc:1434] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.059s user 0.010s sys 0.005s
I20260809 01:45:28.785976 11554 tablet_bootstrap.cc:492] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:28.835824 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 7e79e27a6c44498b99f363dd654fbb7e (DEFAULT_TABLE table=default.Table_2b6ecadf1f6741229c08338a74153bf8 [id=1bdacb31d36c44f58ce3b2264fd8cfcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.837765 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e79e27a6c44498b99f363dd654fbb7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.851919 11589 raft_consensus.cc:493] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:28.852842 11589 raft_consensus.cc:515] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:28.699216 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bdd0ebb4ff348db986bf29913d430ff" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
W20260809 01:45:28.723655 11005 tablet.cc:2406] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:45:28.769174 10877 tablet_service.cc:1511] Processing CreateTablet for tablet 7e79e27a6c44498b99f363dd654fbb7e (DEFAULT_TABLE table=default.Table_2b6ecadf1f6741229c08338a74153bf8 [id=1bdacb31d36c44f58ce3b2264fd8cfcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:28.906172 11578 raft_consensus.cc:493] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:28.908012 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7e79e27a6c44498b99f363dd654fbb7e consensus state has no leader
I20260809 01:45:28.908505 11578 raft_consensus.cc:515] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:28.914601 10595 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet a8c4892ed69143cfb9659a6f4a9d9e55
I20260809 01:45:28.917351 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet a8c4892ed69143cfb9659a6f4a9d9e55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:28.918254 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet a8c4892ed69143cfb9659a6f4a9d9e55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:28.937867 10599 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 9bdd0ebb4ff348db986bf29913d430ff
I20260809 01:45:28.940610 11578 leader_election.cc:290] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:28.945140 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet 9bdd0ebb4ff348db986bf29913d430ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:28.945981 11551 ts_tablet_manager.cc:1434] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.486s user 0.003s sys 0.011s
I20260809 01:45:28.949419 11553 tablet_bootstrap.cc:492] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:28.949613 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e79e27a6c44498b99f363dd654fbb7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:28.953122 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_49db2b3dc5474004946e08a5245687c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:28.952967 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_c46005ef8c674ee4819c91c204485828"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:28.956241 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49db2b3dc5474004946e08a5245687c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:28.950367 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet 9bdd0ebb4ff348db986bf29913d430ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:45982
W20260809 01:45:28.956277 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c46005ef8c674ee4819c91c204485828 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:28.958647 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 949507eb878d470c8cf67abe0291f8eb (table default.Table_04dac09c53c84a28a2f14c19104f5c9d [id=6a7b6884b2e74006a43ce56bc5c264ca]) successfully deleted
I20260809 01:45:28.835518 11570 raft_consensus.cc:515] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:28.962368 10595 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_cad8aab384fb4dd59f29d69bc4798da7" } new_table_name: "default.Table_636659c489a74b76ab839543e432ee5c" modify_external_catalogs: true
I20260809 01:45:28.963413 11570 leader_election.cc:290] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:28.963858 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_ac50f0c1230d4c75a903626ebe80c2fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:28.970608 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ac50f0c1230d4c75a903626ebe80c2fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:28.947257 11551 tablet_bootstrap.cc:492] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:28.991673 11565 ts_tablet_manager.cc:1929] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:28.992377 11565 log.cc:1199] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/949507eb878d470c8cf67abe0291f8eb
W20260809 01:45:28.993971 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 64ac88034c7a4dbb82a39ac7731c37e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 64ac88034c7a4dbb82a39ac7731c37e5 already in progress: creating tablet
I20260809 01:45:28.995055 11589 leader_election.cc:290] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:29.002050 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet 949507eb878d470c8cf67abe0291f8eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:27 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:29.025008 11551 tablet_bootstrap.cc:654] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:29.021999 11590 raft_consensus.cc:493] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:29.035629 11590 raft_consensus.cc:515] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:29.041102 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 7e79e27a6c44498b99f363dd654fbb7e (DEFAULT_TABLE table=default.Table_2b6ecadf1f6741229c08338a74153bf8 [id=1bdacb31d36c44f58ce3b2264fd8cfcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.042279 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e79e27a6c44498b99f363dd654fbb7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.042979 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet a8c4892ed69143cfb9659a6f4a9d9e55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:29.046972 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2b6684972a477e83b19d977f0c1e7c" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:29.048882 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bdd0ebb4ff348db986bf29913d430ff" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:29.049628 11398 raft_consensus.cc:3060] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:29.056988 11551 tablet_bootstrap.cc:492] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:29.057981 11551 ts_tablet_manager.cc:1403] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.111s user 0.019s sys 0.013s
I20260809 01:45:29.064623 11551 raft_consensus.cc:359] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:29.066167 11551 raft_consensus.cc:385] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:29.066766 11551 raft_consensus.cc:740] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:29.066684 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bdd0ebb4ff348db986bf29913d430ff" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:29.077376 11565 ts_tablet_manager.cc:1950] T 949507eb878d470c8cf67abe0291f8eb P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
W20260809 01:45:29.080765 11289 leader_election.cc:343] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:29.081722 11289 leader_election.cc:304] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:29.085027 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c0c5eaa1f64c0298226901de71e7f9" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:29.086102 11160 raft_consensus.cc:2468] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:29.088927 11283 leader_election.cc:304] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
W20260809 01:45:29.107951 10595 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5f8824953c794450ba274b160d891fcb has no consensus state
I20260809 01:45:29.107893 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 9bdd0ebb4ff348db986bf29913d430ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:53476
W20260809 01:45:29.142765 11063 leader_election.cc:343] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:29.060228 11590 leader_election.cc:290] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:28.998319 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2b6684972a477e83b19d977f0c1e7c" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
W20260809 01:45:29.160820 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet a8c4892ed69143cfb9659a6f4a9d9e55 because the tablet was not found. No further retry: Not found: Tablet not found: a8c4892ed69143cfb9659a6f4a9d9e55
W20260809 01:45:29.161700 11063 leader_election.cc:343] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:29.162025 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 9bdd0ebb4ff348db986bf29913d430ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9bdd0ebb4ff348db986bf29913d430ff already in progress: creating tablet
W20260809 01:45:29.163132 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet a8c4892ed69143cfb9659a6f4a9d9e55 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a8c4892ed69143cfb9659a6f4a9d9e55 already in progress: creating tablet
I20260809 01:45:29.165788 10873 tablet_service.cc:1511] Processing CreateTablet for tablet af8056e8875c42948923e1d76e49d6d0 (DEFAULT_TABLE table=default.Table_14857c122d584ff994a4159a0fa99b98 [id=5558f8c7af7a4ad48c8b1ded407d61ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.168267 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 2674c9dbf9e44d63b41427f2fda4e311 (DEFAULT_TABLE table=default.Table_7f7d5bf058bd48ff8e3660ca567a5718 [id=0806119aff0647e7aee1fc802147e2f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.186488 11126 tablet_service.cc:1511] Processing CreateTablet for tablet 2674c9dbf9e44d63b41427f2fda4e311 (DEFAULT_TABLE table=default.Table_7f7d5bf058bd48ff8e3660ca567a5718 [id=0806119aff0647e7aee1fc802147e2f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.188568 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2674c9dbf9e44d63b41427f2fda4e311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.191275 11129 tablet_service.cc:1511] Processing CreateTablet for tablet af8056e8875c42948923e1d76e49d6d0 (DEFAULT_TABLE table=default.Table_14857c122d584ff994a4159a0fa99b98 [id=5558f8c7af7a4ad48c8b1ded407d61ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.193976 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af8056e8875c42948923e1d76e49d6d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.193351 11551 consensus_queue.cc:260] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:29.199138 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 5f8824953c794450ba274b160d891fcb (DEFAULT_TABLE table=default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.204627 11551 ts_tablet_manager.cc:1434] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.146s user 0.017s sys 0.063s
I20260809 01:45:29.206359 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f8824953c794450ba274b160d891fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.206943 11589 raft_consensus.cc:493] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:29.213680 11589 raft_consensus.cc:515] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:29.216274 11564 tablet_replica.cc:333] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:29.179221 10871 tablet_service.cc:1511] Processing CreateTablet for tablet 5f8824953c794450ba274b160d891fcb (DEFAULT_TABLE table=default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.240188 11551 tablet_bootstrap.cc:492] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:29.242331 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af8056e8875c42948923e1d76e49d6d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.246562 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f8824953c794450ba274b160d891fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.251631 11589 leader_election.cc:290] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:29.252640 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ac88034c7a4dbb82a39ac7731c37e5" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:29.253774 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2674c9dbf9e44d63b41427f2fda4e311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.173508 11595 tablet_replica.cc:333] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
W20260809 01:45:29.257957 11063 leader_election.cc:343] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:29.262434 11595 raft_consensus.cc:2243] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260809 01:45:29.269058 11354 consensus_meta.cc:287] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.533s user 0.010s sys 0.003s
W20260809 01:45:29.271462 10804 leader_election.cc:343] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: 949507eb878d470c8cf67abe0291f8eb
W20260809 01:45:29.169466 10867 consensus_meta.cc:287] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent flushing consensus metadata: real 0.519s user 0.005s sys 0.004s
I20260809 01:45:29.301821 11595 raft_consensus.cc:2272] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:29.303951 11601 raft_consensus.cc:2804] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:29.306556 11595 ts_tablet_manager.cc:1916] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:29.310142 11398 raft_consensus.cc:2468] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:29.329708 11590 raft_consensus.cc:2749] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:29.331445 11564 raft_consensus.cc:2243] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:29.331892 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 2674c9dbf9e44d63b41427f2fda4e311 (DEFAULT_TABLE table=default.Table_7f7d5bf058bd48ff8e3660ca567a5718 [id=0806119aff0647e7aee1fc802147e2f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.332979 11554 tablet_bootstrap.cc:700] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316: Time spent opening tablet: real 0.163s user 0.000s sys 0.002s
I20260809 01:45:29.333710 11554 tablet_bootstrap.cc:654] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:29.304591 11601 raft_consensus.cc:493] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 01:45:29.298241 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 205130us
I20260809 01:45:29.346458 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 5) took 1387 ms. Trace:
I20260809 01:45:29.346967 11601 raft_consensus.cc:3060] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:29.347379 11565 rpcz_store.cc:276] 0809 01:45:27.959351 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:27.980703 (+ 21352us) service_pool.cc:224] Handling call
0809 01:45:28.027437 (+ 46734us) ts_tablet_manager.cc:1151] Deleting tablet 949507eb878d470c8cf67abe0291f8eb
0809 01:45:28.027513 (+ 76us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:28.485592 (+458079us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:28.991449 (+505857us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:29.346314 (+354865us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":46008}
I20260809 01:45:29.351183 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 949507eb878d470c8cf67abe0291f8eb (table default.Table_04dac09c53c84a28a2f14c19104f5c9d [id=6a7b6884b2e74006a43ce56bc5c264ca]) successfully deleted
I20260809 01:45:29.352926 11565 tablet_replica.cc:333] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:29.354600 11565 raft_consensus.cc:2243] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:29.348112 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2674c9dbf9e44d63b41427f2fda4e311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.366629 11354 tablet_service.cc:1511] Processing CreateTablet for tablet af8056e8875c42948923e1d76e49d6d0 (DEFAULT_TABLE table=default.Table_14857c122d584ff994a4159a0fa99b98 [id=5558f8c7af7a4ad48c8b1ded407d61ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.381116 11553 tablet_bootstrap.cc:654] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:29.368539 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af8056e8875c42948923e1d76e49d6d0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:29.360306 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 301826us
I20260809 01:45:29.397222 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ac88034c7a4dbb82a39ac7731c37e5" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:29.407244 10599 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet f3c0c5eaa1f64c0298226901de71e7f9 (table default.Table_6149470a609f459c951f42acf3332a0d [id=71f0e1b66e0d466ba1297d4ce6c85ad9]). Expected version 2 got 0
I20260809 01:45:29.427260 11564 raft_consensus.cc:2272] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:29.450775 11369 tablet_service.cc:1511] Processing CreateTablet for tablet 5f8824953c794450ba274b160d891fcb (DEFAULT_TABLE table=default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.452806 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f8824953c794450ba274b160d891fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.463042 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_175d0b46158d49a88aac3833ba31533a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:29.466207 10595 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_14857c122d584ff994a4159a0fa99b98" } new_table_name: "default.Table_90164206f2f44d17895684c1d4dc28da" modify_external_catalogs: true
I20260809 01:45:29.470254 10591 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_b9f38ba904ef48559bddb776d4f3a4ff" } modify_external_catalogs: true
I20260809 01:45:29.471262 10591 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_b9f38ba904ef48559bddb776d4f3a4ff" } modify_external_catalogs: true
I20260809 01:45:29.471776 11564 ts_tablet_manager.cc:1916] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:29.430435 11125 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b2b6684972a477e83b19d977f0c1e7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:27 UTC) from {username='slave'} at 127.0.0.1:56566
W20260809 01:45:29.431406 11062 leader_election.cc:343] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:29.473346 11601 raft_consensus.cc:515] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:29.474311 11062 leader_election.cc:304] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:29.476173 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 44834.2us
I20260809 01:45:29.439752 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 44ac6aecc93447f7935fbd656fd2a86b (DEFAULT_TABLE table=default.Table_ac50f0c1230d4c75a903626ebe80c2fd [id=17cbc8bf77e142d08d03bd7a051e00ad]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:29.470762 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_175d0b46158d49a88aac3833ba31533a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:29.446507 11551 tablet_bootstrap.cc:700] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7: Time spent opening tablet: real 0.125s user 0.000s sys 0.003s
I20260809 01:45:29.478951 10592 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_7f7d5bf058bd48ff8e3660ca567a5718" } modify_external_catalogs: true
I20260809 01:45:29.450240 11123 tablet_service.cc:1511] Processing CreateTablet for tablet c9d839443474498386220c288ff1721b (DEFAULT_TABLE table=default.Table_49db2b3dc5474004946e08a5245687c3 [id=97ccdb35be6a437288222feb89f9e751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.479910 10592 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_7f7d5bf058bd48ff8e3660ca567a5718" } modify_external_catalogs: true
I20260809 01:45:29.477725 11589 raft_consensus.cc:2749] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:29.477495 11062 leader_election.cc:304] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:29.483464 11589 raft_consensus.cc:2804] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:29.484205 11589 raft_consensus.cc:697] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:29.479934 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_3693303b264042b4acfd1cb8907adb78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:29.487586 11589 consensus_queue.cc:237] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:29.490833 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3693303b264042b4acfd1cb8907adb78 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:29.479118 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44ac6aecc93447f7935fbd656fd2a86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.479907 11551 tablet_bootstrap.cc:654] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:29.502097 11123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9d839443474498386220c288ff1721b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.508960 10589 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_d643f50faae24194b5f791d79b9905d3" } new_table_name: "default.Table_a3341f14abab492293b1beadb947e74b" modify_external_catalogs: true
I20260809 01:45:29.510470 11601 leader_election.cc:290] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:29.512653 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c0c5eaa1f64c0298226901de71e7f9" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:29.513501 11160 raft_consensus.cc:3060] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:29.519074 11062 leader_election.cc:343] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:29.521169 11062 leader_election.cc:304] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:29.385808 10867 tablet_service.cc:1511] Processing CreateTablet for tablet 44ac6aecc93447f7935fbd656fd2a86b (DEFAULT_TABLE table=default.Table_ac50f0c1230d4c75a903626ebe80c2fd [id=17cbc8bf77e142d08d03bd7a051e00ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.393962 10870 tablet_service.cc:1511] Processing CreateTablet for tablet c9d839443474498386220c288ff1721b (DEFAULT_TABLE table=default.Table_49db2b3dc5474004946e08a5245687c3 [id=97ccdb35be6a437288222feb89f9e751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.397284 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c0c5eaa1f64c0298226901de71e7f9" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:29.534281 11565 raft_consensus.cc:2272] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:29.534765 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9d839443474498386220c288ff1721b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.538251 11565 ts_tablet_manager.cc:1916] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:29.538398 10867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44ac6aecc93447f7935fbd656fd2a86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.545203 11554 tablet_bootstrap.cc:492] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:29.546586 11554 ts_tablet_manager.cc:1403] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.761s user 0.017s sys 0.015s
I20260809 01:45:29.549700 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c0c5eaa1f64c0298226901de71e7f9" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
W20260809 01:45:29.557108 11289 leader_election.cc:343] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:29.558017 11289 leader_election.cc:343] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:29.544620 11062 raft_consensus.cc:2686] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260809 01:45:29.563339 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 44499.9us
W20260809 01:45:29.567999 10805 leader_election.cc:343] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Not found: Tablet not found: 949507eb878d470c8cf67abe0291f8eb
I20260809 01:45:29.568432 10805 leader_election.cc:304] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:29.569782 10805 raft_consensus.cc:2686] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 01:45:29.570771 11160 raft_consensus.cc:2468] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:29.571965 11283 leader_election.cc:304] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:29.572769 11601 raft_consensus.cc:2804] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:29.572863 10874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45982 (request call id 10) took 1144 ms. Trace:
I20260809 01:45:29.573339 10874 rpcz_store.cc:276] 0809 01:45:28.428642 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:28.429070 (+ 428us) service_pool.cc:224] Handling call
0809 01:45:28.454579 (+ 25509us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:28.454710 (+ 131us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:28.987449 (+532739us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:29.572279 (+584830us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:29.572796 (+ 517us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:29.576907 11601 raft_consensus.cc:697] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:29.578173 11601 consensus_queue.cc:237] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:29.590981 11551 tablet_bootstrap.cc:492] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:29.591674 11551 ts_tablet_manager.cc:1403] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.352s user 0.010s sys 0.017s
I20260809 01:45:29.595242 11554 raft_consensus.cc:359] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:29.596191 11554 raft_consensus.cc:385] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:29.604377 11551 raft_consensus.cc:359] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:29.597245 11554 raft_consensus.cc:740] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:29.605767 11554 consensus_queue.cc:260] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:29.608477 11554 ts_tablet_manager.cc:1434] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.062s user 0.008s sys 0.000s
I20260809 01:45:29.605252 11551 raft_consensus.cc:385] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:29.618422 11551 raft_consensus.cc:740] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:29.631028 11551 consensus_queue.cc:260] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:29.640537 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 44ac6aecc93447f7935fbd656fd2a86b (DEFAULT_TABLE table=default.Table_ac50f0c1230d4c75a903626ebe80c2fd [id=17cbc8bf77e142d08d03bd7a051e00ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.642887 11360 tablet_service.cc:1511] Processing CreateTablet for tablet c9d839443474498386220c288ff1721b (DEFAULT_TABLE table=default.Table_49db2b3dc5474004946e08a5245687c3 [id=97ccdb35be6a437288222feb89f9e751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.644876 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44ac6aecc93447f7935fbd656fd2a86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.648689 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9d839443474498386220c288ff1721b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.649471 11551 ts_tablet_manager.cc:1434] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.057s user 0.007s sys 0.005s
I20260809 01:45:29.609444 11554 tablet_bootstrap.cc:492] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:29.683967 11564 ts_tablet_manager.cc:1929] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:29.684842 11564 log.cc:1199] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/9bdd0ebb4ff348db986bf29913d430ff
I20260809 01:45:29.685922 11564 ts_tablet_manager.cc:1950] T 9bdd0ebb4ff348db986bf29913d430ff P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:29.690798 11551 tablet_bootstrap.cc:492] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
W20260809 01:45:29.697242 10580 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 5f8824953c794450ba274b160d891fcb (table default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5f8824953c794450ba274b160d891fcb consensus state has no leader
I20260809 01:45:29.704936 11565 ts_tablet_manager.cc:1929] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:29.712751 11553 tablet_bootstrap.cc:492] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:29.713441 11553 ts_tablet_manager.cc:1403] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.764s user 0.008s sys 0.016s
I20260809 01:45:29.718143 11565 log.cc:1199] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/64ac88034c7a4dbb82a39ac7731c37e5
I20260809 01:45:29.718981 11565 ts_tablet_manager.cc:1950] T 64ac88034c7a4dbb82a39ac7731c37e5 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:29.721324 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 8) took 1449 ms. Trace:
I20260809 01:45:29.721753 11565 rpcz_store.cc:276] 0809 01:45:28.272035 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:28.272224 (+ 189us) service_pool.cc:224] Handling call
0809 01:45:29.352628 (+1080404us) ts_tablet_manager.cc:1151] Deleting tablet 64ac88034c7a4dbb82a39ac7731c37e5
0809 01:45:29.352703 (+ 75us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:29.624753 (+272050us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:29.704805 (+ 80052us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:29.721249 (+ 16444us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1079680}
I20260809 01:45:29.723300 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 64ac88034c7a4dbb82a39ac7731c37e5 (table default.Table_aaad8afaf8134c808577f0b370975c6c [id=7ff740ce8d4f4b5995b4d5ef52cafada]) successfully deleted
I20260809 01:45:29.730057 11554 tablet_bootstrap.cc:654] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:29.734165 11551 tablet_bootstrap.cc:654] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:29.744838 10596 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet f3c0c5eaa1f64c0298226901de71e7f9 (table default.Table_6149470a609f459c951f42acf3332a0d [id=71f0e1b66e0d466ba1297d4ce6c85ad9]). Expected version 2 got 0
I20260809 01:45:29.745956 11553 raft_consensus.cc:359] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:29.746902 11553 raft_consensus.cc:385] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:29.747418 11553 raft_consensus.cc:740] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:29.749598 10873 tablet_service.cc:1511] Processing CreateTablet for tablet 61c83b8bcf54461bb24565e3a989100a (DEFAULT_TABLE table=default.Table_c46005ef8c674ee4819c91c204485828 [id=8b42fb7f9413434b96d5e788d9919ec2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.751626 11565 tablet_replica.cc:333] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:29.752985 11565 raft_consensus.cc:2243] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:29.754371 11565 pending_rounds.cc:70] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7: Trying to abort 1 pending ops.
I20260809 01:45:29.754807 11565 pending_rounds.cc:77] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438751322611712 op_type: NO_OP noop_request { }
I20260809 01:45:29.755496 11565 raft_consensus.cc:2889] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:29.756165 11565 raft_consensus.cc:2272] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:29.757287 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61c83b8bcf54461bb24565e3a989100a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.748543 11553 consensus_queue.cc:260] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:29.777670 11565 ts_tablet_manager.cc:1916] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:29.790400 11551 tablet_bootstrap.cc:492] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:29.791002 11551 ts_tablet_manager.cc:1403] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.101s user 0.008s sys 0.011s
W20260809 01:45:29.796267 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet a8c4892ed69143cfb9659a6f4a9d9e55 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a8c4892ed69143cfb9659a6f4a9d9e55 already in progress: creating tablet
I20260809 01:45:29.765280 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 9bdd0ebb4ff348db986bf29913d430ff (table default.Table_ecca25c58193453e8cf32461e3af95a9 [id=ff18a4c4305e465caf0a5801ccbee7a1]) successfully deleted
I20260809 01:45:29.800787 11123 tablet_service.cc:1511] Processing CreateTablet for tablet 61c83b8bcf54461bb24565e3a989100a (DEFAULT_TABLE table=default.Table_c46005ef8c674ee4819c91c204485828 [id=8b42fb7f9413434b96d5e788d9919ec2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.802776 11123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61c83b8bcf54461bb24565e3a989100a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.826607 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_84e4a5312bbb43c9bd562d0f742537e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:29.830492 11553 ts_tablet_manager.cc:1434] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.117s user 0.016s sys 0.008s
I20260809 01:45:29.832023 11553 tablet_bootstrap.cc:492] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:29.834860 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_71d1332c30c4432791d1a84d52510063"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:29.840677 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_71d1332c30c4432791d1a84d52510063 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:29.842041 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_84e4a5312bbb43c9bd562d0f742537e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:29.857672 11553 tablet_bootstrap.cc:654] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:29.866047 10588 catalog_manager.cc:5719] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67). New cstate: current_term: 1 leader_uuid: "4f60c9ac9414403d9ea5a97b103c2316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:29.867959 10588 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet f3c0c5eaa1f64c0298226901de71e7f9 (table default.Table_6149470a609f459c951f42acf3332a0d [id=71f0e1b66e0d466ba1297d4ce6c85ad9]). Expected version 2 got 0
I20260809 01:45:29.870508 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 61c83b8bcf54461bb24565e3a989100a (DEFAULT_TABLE table=default.Table_c46005ef8c674ee4819c91c204485828 [id=8b42fb7f9413434b96d5e788d9919ec2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:29.870843 11553 tablet_bootstrap.cc:492] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:29.871225 11553 ts_tablet_manager.cc:1403] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.014s
I20260809 01:45:29.871522 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61c83b8bcf54461bb24565e3a989100a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:29.876425 11553 raft_consensus.cc:359] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:29.877197 11553 raft_consensus.cc:385] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:29.877507 11553 raft_consensus.cc:740] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:29.878369 11551 raft_consensus.cc:359] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:29.879086 11551 raft_consensus.cc:385] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:29.879383 11551 raft_consensus.cc:740] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:29.881394 11553 consensus_queue.cc:260] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:29.883775 11553 ts_tablet_manager.cc:1434] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.012s user 0.005s sys 0.002s
I20260809 01:45:29.889055 11553 tablet_bootstrap.cc:492] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:29.900257 11554 tablet_bootstrap.cc:492] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:29.900755 11554 ts_tablet_manager.cc:1403] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.292s user 0.031s sys 0.029s
I20260809 01:45:29.902360 11551 consensus_queue.cc:260] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:29.903519 11554 raft_consensus.cc:359] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:29.904181 11554 raft_consensus.cc:385] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:29.904469 11554 raft_consensus.cc:740] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:29.905238 11554 consensus_queue.cc:260] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:29.910184 11553 tablet_bootstrap.cc:654] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:29.911512 11554 ts_tablet_manager.cc:1434] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260809 01:45:29.916642 11554 tablet_bootstrap.cc:492] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:29.922447 11554 tablet_bootstrap.cc:654] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:29.930480 11553 tablet_bootstrap.cc:492] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:29.930897 11553 ts_tablet_manager.cc:1403] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.042s user 0.007s sys 0.005s
I20260809 01:45:29.933849 11553 raft_consensus.cc:359] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:29.934566 11553 raft_consensus.cc:385] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:29.934859 11553 raft_consensus.cc:740] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:29.935647 11553 consensus_queue.cc:260] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:29.939543 11553 ts_tablet_manager.cc:1434] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
I20260809 01:45:29.940549 11553 tablet_bootstrap.cc:492] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:29.943692 11595 ts_tablet_manager.cc:1929] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:29.944192 11595 log.cc:1199] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/949507eb878d470c8cf67abe0291f8eb
I20260809 01:45:29.950941 11554 tablet_bootstrap.cc:492] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:29.951337 11554 ts_tablet_manager.cc:1403] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.006s
I20260809 01:45:29.952644 11551 ts_tablet_manager.cc:1434] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.161s user 0.009s sys 0.000s
I20260809 01:45:29.953869 11554 raft_consensus.cc:359] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:29.954500 11554 raft_consensus.cc:385] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:29.954753 11554 raft_consensus.cc:740] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:29.955623 11554 consensus_queue.cc:260] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:29.961118 11565 ts_tablet_manager.cc:1929] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:29.964720 11551 tablet_bootstrap.cc:492] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:29.966079 11554 ts_tablet_manager.cc:1434] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
I20260809 01:45:29.970142 11554 tablet_bootstrap.cc:492] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:29.949503 11553 tablet_bootstrap.cc:654] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:29.987299 11554 tablet_bootstrap.cc:654] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:29.992654 11551 tablet_bootstrap.cc:654] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:30.001175 10595 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet af8056e8875c42948923e1d76e49d6d0 consensus state has no leader
I20260809 01:45:30.001897 11565 log.cc:1199] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/9bdd0ebb4ff348db986bf29913d430ff
I20260809 01:45:30.002403 11553 tablet_bootstrap.cc:492] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:30.002877 11553 ts_tablet_manager.cc:1403] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.063s user 0.014s sys 0.005s
I20260809 01:45:30.003057 11565 ts_tablet_manager.cc:1950] T 9bdd0ebb4ff348db986bf29913d430ff P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:30.003933 10599 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet dfb4ad36ca954c7c99407612c0f30711 (table default.Table_d643f50faae24194b5f791d79b9905d3 [id=7831718cddee4b5b9c42cec19ff7d074]). Expected version 1 got 0
I20260809 01:45:30.007378 11553 raft_consensus.cc:359] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.008149 11553 raft_consensus.cc:385] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:30.008203 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 16) took 1063 ms. Trace:
I20260809 01:45:30.008522 11553 raft_consensus.cc:740] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:30.008688 11565 rpcz_store.cc:276] 0809 01:45:28.944617 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:28.944863 (+ 246us) service_pool.cc:224] Handling call
0809 01:45:29.751372 (+806509us) ts_tablet_manager.cc:1151] Deleting tablet 9bdd0ebb4ff348db986bf29913d430ff
0809 01:45:29.751447 (+ 75us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:29.953761 (+202314us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:29.960990 (+ 7229us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:30.008111 (+ 47121us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":805612}
I20260809 01:45:30.010154 11553 consensus_queue.cc:260] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.011526 10867 tablet_service.cc:1558] Processing DeleteTablet for tablet 9bdd0ebb4ff348db986bf29913d430ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:30.015918 11553 ts_tablet_manager.cc:1434] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.013s user 0.004s sys 0.006s
I20260809 01:45:30.017489 11553 tablet_bootstrap.cc:492] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:30.023985 11554 tablet_bootstrap.cc:492] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:30.024343 11554 ts_tablet_manager.cc:1403] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.054s user 0.009s sys 0.010s
I20260809 01:45:30.027788 11554 raft_consensus.cc:359] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.028695 11554 raft_consensus.cc:385] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:30.029306 11554 raft_consensus.cc:740] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:30.030236 11554 consensus_queue.cc:260] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.038535 11554 ts_tablet_manager.cc:1434] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.014s user 0.002s sys 0.006s
I20260809 01:45:30.049652 11554 tablet_bootstrap.cc:492] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
W20260809 01:45:30.059131 10589 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dfb4ad36ca954c7c99407612c0f30711 consensus state has no leader
I20260809 01:45:30.066885 11553 tablet_bootstrap.cc:654] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:30.068890 10592 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 2674c9dbf9e44d63b41427f2fda4e311
I20260809 01:45:30.076731 11554 tablet_bootstrap.cc:654] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:30.080931 10591 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 7e79e27a6c44498b99f363dd654fbb7e
I20260809 01:45:30.087189 10867 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e79e27a6c44498b99f363dd654fbb7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:29 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:30.092316 11354 tablet_service.cc:1558] Processing DeleteTablet for tablet 2674c9dbf9e44d63b41427f2fda4e311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:29 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:30.092558 11624 raft_consensus.cc:493] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:30.093078 11624 raft_consensus.cc:515] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.094511 11354 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e79e27a6c44498b99f363dd654fbb7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:29 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:30.095181 11624 leader_election.cc:290] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:30.097569 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2b6684972a477e83b19d977f0c1e7c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:30.097834 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 2674c9dbf9e44d63b41427f2fda4e311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:29 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:30.098505 11398 raft_consensus.cc:2468] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 0.
I20260809 01:45:30.100816 10805 leader_election.cc:304] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:30.103008 11624 raft_consensus.cc:2804] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:30.103586 11624 raft_consensus.cc:493] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:30.104260 11624 raft_consensus.cc:3060] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:29.945338 11595 ts_tablet_manager.cc:1950] T 949507eb878d470c8cf67abe0291f8eb P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:30.123893 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_a1852e7ed9bd44cd998c06eaf486549a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:30.126446 11554 tablet_bootstrap.cc:492] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:30.126886 11554 ts_tablet_manager.cc:1403] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.082s user 0.008s sys 0.011s
W20260809 01:45:30.128674 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a1852e7ed9bd44cd998c06eaf486549a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:30.128432 11624 raft_consensus.cc:515] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.130705 11554 raft_consensus.cc:359] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.131443 11554 raft_consensus.cc:385] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 01:45:30.117192 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 2674c9dbf9e44d63b41427f2fda4e311 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2674c9dbf9e44d63b41427f2fda4e311 already in progress: creating tablet
I20260809 01:45:30.131722 11554 raft_consensus.cc:740] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
W20260809 01:45:30.132706 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 7e79e27a6c44498b99f363dd654fbb7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e79e27a6c44498b99f363dd654fbb7e already in progress: creating tablet
I20260809 01:45:30.136716 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2b6684972a477e83b19d977f0c1e7c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:30.075217 11123 tablet_service.cc:1558] Processing DeleteTablet for tablet 2674c9dbf9e44d63b41427f2fda4e311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:29 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:30.137419 11398 raft_consensus.cc:3060] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:30.085507 11124 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e79e27a6c44498b99f363dd654fbb7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:29 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:30.104898 11589 raft_consensus.cc:493] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:30.139199 11589 raft_consensus.cc:515] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.139487 11554 consensus_queue.cc:260] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.142793 11554 ts_tablet_manager.cc:1434] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20260809 01:45:30.143776 11554 tablet_bootstrap.cc:492] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:30.147713 11398 raft_consensus.cc:2468] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 1.
I20260809 01:45:30.151868 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "825c568f98014fbf92ade7bd99763581" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:30.152853 10910 raft_consensus.cc:2468] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:30.153061 11554 tablet_bootstrap.cc:654] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:30.153568 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "825c568f98014fbf92ade7bd99763581" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:30.154434 11395 raft_consensus.cc:2468] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:30.154844 10805 leader_election.cc:304] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:30.156344 11062 leader_election.cc:304] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:30.158411 11589 leader_election.cc:290] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:30.159228 11589 raft_consensus.cc:2804] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:30.159261 11624 leader_election.cc:290] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:30.159619 11589 raft_consensus.cc:493] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:30.159970 11589 raft_consensus.cc:3060] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:30.160181 11624 raft_consensus.cc:2804] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:30.160992 11624 raft_consensus.cc:697] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Becoming Leader. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Running, Role: LEADER
I20260809 01:45:30.108105 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2b6684972a477e83b19d977f0c1e7c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:30.132839 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2b6684972a477e83b19d977f0c1e7c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:30.162642 11159 raft_consensus.cc:3060] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:30.162175 11624 consensus_queue.cc:237] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.184437 11554 tablet_bootstrap.cc:492] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:30.184916 11554 ts_tablet_manager.cc:1403] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.041s user 0.009s sys 0.010s
I20260809 01:45:30.196904 11159 raft_consensus.cc:2468] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 1.
I20260809 01:45:30.193133 11554 raft_consensus.cc:359] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.200678 11554 raft_consensus.cc:385] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:30.200995 11554 raft_consensus.cc:740] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:30.201805 11554 consensus_queue.cc:260] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.203069 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 9bdd0ebb4ff348db986bf29913d430ff (table default.Table_ecca25c58193453e8cf32461e3af95a9 [id=ff18a4c4305e465caf0a5801ccbee7a1]) successfully deleted
I20260809 01:45:30.204351 11565 tablet_replica.cc:333] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:30.205447 11565 raft_consensus.cc:2243] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:30.206195 11565 raft_consensus.cc:2272] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:30.213357 11554 ts_tablet_manager.cc:1434] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.024s user 0.005s sys 0.003s
I20260809 01:45:30.214407 11554 tablet_bootstrap.cc:492] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:30.217376 10592 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_3693303b264042b4acfd1cb8907adb78" } new_table_name: "default.Table_521c14a29ad84d2b99cdf431e3c63eac" modify_external_catalogs: true
I20260809 01:45:30.246291 11554 tablet_bootstrap.cc:654] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:30.251863 11595 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 17) took 1276 ms. Trace:
I20260809 01:45:30.252432 11595 rpcz_store.cc:276] 0809 01:45:28.975655 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:28.997945 (+ 22290us) service_pool.cc:224] Handling call
0809 01:45:29.141672 (+143727us) ts_tablet_manager.cc:1151] Deleting tablet 949507eb878d470c8cf67abe0291f8eb
0809 01:45:29.141741 (+ 69us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:29.678609 (+536868us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:29.943536 (+264927us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:30.251650 (+308114us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":132352}
I20260809 01:45:30.254832 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 949507eb878d470c8cf67abe0291f8eb (table default.Table_04dac09c53c84a28a2f14c19104f5c9d [id=6a7b6884b2e74006a43ce56bc5c264ca]) successfully deleted
I20260809 01:45:30.261286 11554 tablet_bootstrap.cc:492] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:30.261768 11554 ts_tablet_manager.cc:1403] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.048s user 0.014s sys 0.009s
I20260809 01:45:30.268951 11554 raft_consensus.cc:359] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.269809 11554 raft_consensus.cc:385] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:30.270135 11554 raft_consensus.cc:740] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:30.270924 11554 consensus_queue.cc:260] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.272831 10589 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_ac50f0c1230d4c75a903626ebe80c2fd" } new_table_name: "default.Table_9081f03ef3eb4bb49093311263bb3976" modify_external_catalogs: true
I20260809 01:45:30.273377 11554 ts_tablet_manager.cc:1434] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.011s user 0.004s sys 0.003s
I20260809 01:45:30.274376 11554 tablet_bootstrap.cc:492] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:30.277318 11553 tablet_bootstrap.cc:492] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:30.278013 11553 ts_tablet_manager.cc:1403] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.261s user 0.023s sys 0.036s
I20260809 01:45:30.278597 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_be8856c44b864dd6be36f611dfaf4769"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:30.285882 11595 tablet_replica.cc:333] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:30.287175 11595 raft_consensus.cc:2243] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260809 01:45:30.281455 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be8856c44b864dd6be36f611dfaf4769 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:30.287870 11595 raft_consensus.cc:2272] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:30.296414 11553 raft_consensus.cc:359] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.297307 11553 raft_consensus.cc:385] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:30.297676 11553 raft_consensus.cc:740] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:30.304152 11595 ts_tablet_manager.cc:1916] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:30.306465 11553 consensus_queue.cc:260] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:30.307168 10592 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 43d84f310d8e4df7b8b2d602289025ae has no consensus state
I20260809 01:45:30.328816 11553 ts_tablet_manager.cc:1434] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.050s user 0.007s sys 0.002s
I20260809 01:45:30.336884 11360 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b2b6684972a477e83b19d977f0c1e7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:27 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:30.339335 11564 tablet_replica.cc:333] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:30.340806 11564 raft_consensus.cc:2243] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:30.341605 11564 raft_consensus.cc:2272] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:30.343333 11553 tablet_bootstrap.cc:492] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:30.343567 10597 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_006a6ebb9836470396c69b4b78abc954" } modify_external_catalogs: true
I20260809 01:45:30.344334 10597 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_006a6ebb9836470396c69b4b78abc954" } modify_external_catalogs: true
I20260809 01:45:30.351841 11554 tablet_bootstrap.cc:654] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:30.370363 10588 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_6149470a609f459c951f42acf3332a0d" } modify_external_catalogs: true
I20260809 01:45:30.371384 10588 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_6149470a609f459c951f42acf3332a0d" } modify_external_catalogs: true
I20260809 01:45:30.378511 11564 ts_tablet_manager.cc:1916] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:30.379778 11565 ts_tablet_manager.cc:1916] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:30.382408 11642 raft_consensus.cc:493] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:30.383096 11642 raft_consensus.cc:515] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.377136 11589 raft_consensus.cc:515] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.388804 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb4ad36ca954c7c99407612c0f30711" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:30.389637 10910 raft_consensus.cc:2468] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:30.390378 11553 tablet_bootstrap.cc:654] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:30.390959 11589 leader_election.cc:290] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:30.391176 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "825c568f98014fbf92ade7bd99763581" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:30.391912 10908 raft_consensus.cc:3060] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:30.394896 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "825c568f98014fbf92ade7bd99763581" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:30.395656 11395 raft_consensus.cc:3060] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:30.397964 11063 leader_election.cc:304] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:30.404246 11551 tablet_bootstrap.cc:492] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:30.404781 11551 ts_tablet_manager.cc:1403] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.440s user 0.019s sys 0.027s
I20260809 01:45:30.397274 11595 ts_tablet_manager.cc:1929] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:30.409848 11595 log.cc:1199] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/9bdd0ebb4ff348db986bf29913d430ff
I20260809 01:45:30.410738 11595 ts_tablet_manager.cc:1950] T 9bdd0ebb4ff348db986bf29913d430ff P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:30.417665 11553 tablet_bootstrap.cc:492] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:30.418062 11553 ts_tablet_manager.cc:1403] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.075s user 0.010s sys 0.016s
I20260809 01:45:30.418087 11589 raft_consensus.cc:2804] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:30.418503 11589 raft_consensus.cc:493] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:30.419152 11589 raft_consensus.cc:3060] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:30.421977 11553 raft_consensus.cc:359] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.422701 11553 raft_consensus.cc:385] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:30.423046 11553 raft_consensus.cc:740] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:30.427919 11551 raft_consensus.cc:359] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.428571 11551 raft_consensus.cc:385] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:30.428815 11551 raft_consensus.cc:740] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:30.431649 11642 leader_election.cc:290] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:30.433686 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb4ad36ca954c7c99407612c0f30711" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:30.434208 11553 consensus_queue.cc:260] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.434508 11398 raft_consensus.cc:2468] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:30.443512 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 9bdd0ebb4ff348db986bf29913d430ff (table default.Table_ecca25c58193453e8cf32461e3af95a9 [id=ff18a4c4305e465caf0a5801ccbee7a1]) successfully deleted
I20260809 01:45:30.443924 11551 consensus_queue.cc:260] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:30.444365 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 7e79e27a6c44498b99f363dd654fbb7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e79e27a6c44498b99f363dd654fbb7e already in progress: creating tablet
I20260809 01:45:30.444602 10910 raft_consensus.cc:3060] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:30.446147 11551 ts_tablet_manager.cc:1434] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.041s user 0.008s sys 0.001s
W20260809 01:45:30.446450 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 2674c9dbf9e44d63b41427f2fda4e311 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2674c9dbf9e44d63b41427f2fda4e311 already in progress: creating tablet
I20260809 01:45:30.447145 11551 tablet_bootstrap.cc:492] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:30.448308 11553 ts_tablet_manager.cc:1434] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.030s user 0.007s sys 0.000s
I20260809 01:45:30.453794 11553 tablet_bootstrap.cc:492] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:30.466375 11589 raft_consensus.cc:515] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.469215 11589 leader_election.cc:290] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:30.480980 11553 tablet_bootstrap.cc:654] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:30.482512 10589 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 44ac6aecc93447f7935fbd656fd2a86b consensus state has no leader
I20260809 01:45:30.488668 10586 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet f3c0c5eaa1f64c0298226901de71e7f9 (table default.Table_6149470a609f459c951f42acf3332a0d [id=71f0e1b66e0d466ba1297d4ce6c85ad9]). Expected version 2 got 0
I20260809 01:45:30.475162 11551 tablet_bootstrap.cc:654] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:30.492049 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 43d84f310d8e4df7b8b2d602289025ae (DEFAULT_TABLE table=default.Table_521c14a29ad84d2b99cdf431e3c63eac [id=2a521d855351481299fb19d2b1f99b37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:30.493655 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43d84f310d8e4df7b8b2d602289025ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:30.495570 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_4eef5dd402bc41168cd7b02c32e84835"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:30.498600 11601 raft_consensus.cc:493] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:30.499065 11601 raft_consensus.cc:515] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.501236 11601 leader_election.cc:290] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:30.505599 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b7459ad0974cb9948d647459b22260" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:30.505601 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb4ad36ca954c7c99407612c0f30711" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:30.506520 10905 raft_consensus.cc:3060] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:30.503414 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb4ad36ca954c7c99407612c0f30711" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:30.508524 11398 raft_consensus.cc:3060] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:30.508694 11283 leader_election.cc:343] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:30.509369 11551 tablet_bootstrap.cc:492] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:30.509806 11551 ts_tablet_manager.cc:1403] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.063s user 0.014s sys 0.000s
I20260809 01:45:30.507885 10903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b7459ad0974cb9948d647459b22260" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:30.511906 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 43d84f310d8e4df7b8b2d602289025ae (DEFAULT_TABLE table=default.Table_521c14a29ad84d2b99cdf431e3c63eac [id=2a521d855351481299fb19d2b1f99b37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:30.513890 10873 tablet_service.cc:1511] Processing CreateTablet for tablet 43d84f310d8e4df7b8b2d602289025ae (DEFAULT_TABLE table=default.Table_521c14a29ad84d2b99cdf431e3c63eac [id=2a521d855351481299fb19d2b1f99b37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:30.514787 11551 raft_consensus.cc:359] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.515416 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43d84f310d8e4df7b8b2d602289025ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:30.515411 11551 raft_consensus.cc:385] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:30.515661 11551 raft_consensus.cc:740] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:30.519435 10598 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_521c14a29ad84d2b99cdf431e3c63eac" } new_table_name: "default.Table_2712961a5fbc40689aeb5e6dcbf6d79f" modify_external_catalogs: true
I20260809 01:45:30.524776 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43d84f310d8e4df7b8b2d602289025ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:30.527446 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_d5bd8c94a186431ba5f5b8fa23d75b8b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:30.529191 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d5bd8c94a186431ba5f5b8fa23d75b8b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:30.527585 11551 consensus_queue.cc:260] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.537454 11551 ts_tablet_manager.cc:1434] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.027s user 0.008s sys 0.002s
W20260809 01:45:30.544314 11289 leader_election.cc:343] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:30.544143 11551 tablet_bootstrap.cc:492] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:30.544947 11289 leader_election.cc:304] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:30.547886 11601 raft_consensus.cc:2749] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:30.497332 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4eef5dd402bc41168cd7b02c32e84835 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:30.553830 11589 raft_consensus.cc:493] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:30.552516 11601 consensus_queue.cc:1048] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 01:45:30.554561 11589 raft_consensus.cc:515] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.561444 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c4892ed69143cfb9659a6f4a9d9e55" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:30.562539 11392 raft_consensus.cc:2468] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:30.564374 11062 leader_election.cc:304] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:30.594151 11553 tablet_bootstrap.cc:492] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:30.594676 11553 ts_tablet_manager.cc:1403] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.141s user 0.012s sys 0.007s
I20260809 01:45:30.595513 11551 tablet_bootstrap.cc:654] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:30.596439 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_6df59952b936458893928a28040decda"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:30.599119 11642 raft_consensus.cc:2804] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:30.559784 11589 leader_election.cc:290] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:30.599573 11553 raft_consensus.cc:359] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.600314 11553 raft_consensus.cc:385] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:30.600622 11553 raft_consensus.cc:740] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:30.599539 11642 raft_consensus.cc:493] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:30.602371 11642 raft_consensus.cc:3060] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:30.604511 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6df59952b936458893928a28040decda in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:30.602645 10903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c4892ed69143cfb9659a6f4a9d9e55" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
W20260809 01:45:30.606698 11063 leader_election.cc:343] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:30.607856 11554 tablet_bootstrap.cc:492] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:30.608258 11554 ts_tablet_manager.cc:1403] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.334s user 0.004s sys 0.013s
I20260809 01:45:30.601405 11553 consensus_queue.cc:260] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.612315 11553 ts_tablet_manager.cc:1434] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20260809 01:45:30.614152 11554 raft_consensus.cc:359] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.615310 11554 raft_consensus.cc:385] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:30.615583 11554 raft_consensus.cc:740] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:30.617381 11554 consensus_queue.cc:260] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.623652 11554 ts_tablet_manager.cc:1434] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.015s user 0.001s sys 0.007s
I20260809 01:45:30.627277 11553 tablet_bootstrap.cc:492] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:30.633006 11398 raft_consensus.cc:2468] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
W20260809 01:45:30.647490 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 83246us
I20260809 01:45:30.651516 11395 raft_consensus.cc:2468] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:30.652066 10908 raft_consensus.cc:2468] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:30.653901 10588 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet f3c0c5eaa1f64c0298226901de71e7f9
I20260809 01:45:30.660918 11062 leader_election.cc:304] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:30.663275 11656 raft_consensus.cc:2804] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:30.665273 11123 tablet_service.cc:1558] Processing DeleteTablet for tablet f3c0c5eaa1f64c0298226901de71e7f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:56566
W20260809 01:45:30.673600 10598 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 43d84f310d8e4df7b8b2d602289025ae consensus state has no leader
I20260809 01:45:30.656879 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet f3c0c5eaa1f64c0298226901de71e7f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:30.679878 11595 tablet_replica.cc:333] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:30.684700 10596 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet dfb4ad36ca954c7c99407612c0f30711 (table default.Table_a3341f14abab492293b1beadb947e74b [id=7831718cddee4b5b9c42cec19ff7d074]). Expected version 2 got 0
I20260809 01:45:30.687381 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 825c568f98014fbf92ade7bd99763581
I20260809 01:45:30.693728 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 825c568f98014fbf92ade7bd99763581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:30.660954 10905 raft_consensus.cc:2468] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
W20260809 01:45:30.698997 10580 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 44ac6aecc93447f7935fbd656fd2a86b (table default.Table_9081f03ef3eb4bb49093311263bb3976 [id=17cbc8bf77e142d08d03bd7a051e00ad], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 44ac6aecc93447f7935fbd656fd2a86b consensus state has no leader
I20260809 01:45:30.701545 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 9bdd0ebb4ff348db986bf29913d430ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:30.703256 10591 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_a1852e7ed9bd44cd998c06eaf486549a" } new_table_name: "default.Table_e212a7cfad494f2d97ac5eada6ac15f5" modify_external_catalogs: true
I20260809 01:45:30.714884 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b2b6684972a477e83b19d977f0c1e7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:27 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:30.723268 11063 leader_election.cc:304] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
W20260809 01:45:30.726938 10581 consensus_peers.cc:597] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 -> Peer 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063): Couldn't send request to peer 71bc72deb1d2402cabd817dfe3b27e88. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260809 01:45:30.727813 11123 tablet_service.cc:1558] Processing DeleteTablet for tablet 825c568f98014fbf92ade7bd99763581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:30.730788 11589 raft_consensus.cc:2804] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:30.730715 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_25ec70c0a19d4c7dacff1c3a39e55e7e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:30.731446 11589 raft_consensus.cc:697] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:30.733309 11589 consensus_queue.cc:237] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:30.753763 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25ec70c0a19d4c7dacff1c3a39e55e7e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:30.759830 11553 tablet_bootstrap.cc:654] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:30.794358 10591 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 98bee541fc1345fa83ffebeba05d8c8e has no consensus state
I20260809 01:45:30.659276 11354 tablet_service.cc:1558] Processing DeleteTablet for tablet f3c0c5eaa1f64c0298226901de71e7f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:30.665999 11554 tablet_bootstrap.cc:492] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:30.701278 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 825c568f98014fbf92ade7bd99763581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:30.780581 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet 64ac88034c7a4dbb82a39ac7731c37e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:30.791235 11655 raft_consensus.cc:493] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:30.830500 11655 raft_consensus.cc:515] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.833004 11655 leader_election.cc:290] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:30.836014 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ac88034c7a4dbb82a39ac7731c37e5" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:30.836366 11554 tablet_bootstrap.cc:654] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:30.836687 10905 raft_consensus.cc:2468] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:30.838624 11289 leader_election.cc:304] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:30.840193 11601 raft_consensus.cc:493] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:30.840698 11601 raft_consensus.cc:515] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.842350 11624 raft_consensus.cc:493] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:30.843043 11624 raft_consensus.cc:515] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.843900 11601 leader_election.cc:290] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:30.844491 11601 raft_consensus.cc:2804] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:30.844893 11601 raft_consensus.cc:493] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:30.845609 11595 raft_consensus.cc:2243] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:30.848004 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b4850de7d740d794283ca93152fdc1" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
W20260809 01:45:30.850147 11289 leader_election.cc:343] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:30.845333 11601 raft_consensus.cc:3060] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:30.854089 11595 raft_consensus.cc:2272] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:30.854583 11624 leader_election.cc:290] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:30.855479 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c0c5eaa1f64c0298226901de71e7f9" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:30.859578 11595 ts_tablet_manager.cc:1916] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:30.862335 11553 tablet_bootstrap.cc:492] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:30.862756 11553 ts_tablet_manager.cc:1403] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.236s user 0.009s sys 0.006s
I20260809 01:45:30.866616 10586 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_2712961a5fbc40689aeb5e6dcbf6d79f" } modify_external_catalogs: true
I20260809 01:45:30.867324 10586 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_2712961a5fbc40689aeb5e6dcbf6d79f" } modify_external_catalogs: true
I20260809 01:45:30.867514 11553 raft_consensus.cc:359] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.868181 11553 raft_consensus.cc:385] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:30.868480 11553 raft_consensus.cc:740] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:30.870880 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet a8c4892ed69143cfb9659a6f4a9d9e55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:30.804917 11656 raft_consensus.cc:697] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:30.869168 11553 consensus_queue.cc:260] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:30.878312 11283 leader_election.cc:343] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: 64ac88034c7a4dbb82a39ac7731c37e5
I20260809 01:45:30.877004 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b4850de7d740d794283ca93152fdc1" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:30.881812 11553 ts_tablet_manager.cc:1434] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.019s user 0.009s sys 0.000s
I20260809 01:45:30.886094 11642 raft_consensus.cc:515] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.889712 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c4892ed69143cfb9659a6f4a9d9e55" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:30.890569 11553 tablet_bootstrap.cc:492] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:30.891327 11565 ts_tablet_manager.cc:1929] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:30.891857 11565 log.cc:1199] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/5b2b6684972a477e83b19d977f0c1e7c
I20260809 01:45:30.892702 11565 ts_tablet_manager.cc:1950] T 5b2b6684972a477e83b19d977f0c1e7c P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
W20260809 01:45:30.892731 11063 leader_election.cc:343] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:30.902645 11642 leader_election.cc:290] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:30.905545 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c0c5eaa1f64c0298226901de71e7f9" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:30.907991 10804 leader_election.cc:304] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:30.908912 10804 raft_consensus.cc:2686] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 01:45:30.910640 11553 tablet_bootstrap.cc:654] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:30.870697 11601 raft_consensus.cc:515] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.914202 11601 leader_election.cc:290] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
W20260809 01:45:30.915575 11283 leader_election.cc:343] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:30.918728 10592 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_90164206f2f44d17895684c1d4dc28da" } modify_external_catalogs: true
I20260809 01:45:30.916283 11283 leader_election.cc:304] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:30.919185 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c4892ed69143cfb9659a6f4a9d9e55" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:30.919549 10592 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_90164206f2f44d17895684c1d4dc28da" } modify_external_catalogs: true
I20260809 01:45:30.919788 11395 raft_consensus.cc:3060] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:30.920682 11601 raft_consensus.cc:2749] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:30.929967 10596 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_be8856c44b864dd6be36f611dfaf4769" } new_table_name: "default.Table_7033d361a8614225b9dcf10aa47a5bd9" modify_external_catalogs: true
I20260809 01:45:30.937289 11554 tablet_bootstrap.cc:492] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:30.943385 11554 ts_tablet_manager.cc:1403] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.278s user 0.013s sys 0.003s
I20260809 01:45:30.948453 11601 raft_consensus.cc:493] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:30.949219 11655 consensus_queue.cc:1048] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:30.950445 11553 tablet_bootstrap.cc:492] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:30.950881 11553 ts_tablet_manager.cc:1403] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.061s user 0.025s sys 0.010s
I20260809 01:45:30.953050 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_f860a62234ce42b8ae4c4ff3430df44c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:30.954250 11564 ts_tablet_manager.cc:1929] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:30.954869 11564 log.cc:1199] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/5b2b6684972a477e83b19d977f0c1e7c
I20260809 01:45:30.955835 11564 ts_tablet_manager.cc:1950] T 5b2b6684972a477e83b19d977f0c1e7c P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
W20260809 01:45:30.956036 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f860a62234ce42b8ae4c4ff3430df44c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:30.956501 11395 raft_consensus.cc:2468] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:30.948977 11601 raft_consensus.cc:515] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.960330 11601 leader_election.cc:290] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:30.960696 11553 raft_consensus.cc:359] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.961004 11641 raft_consensus.cc:493] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:30.961578 11553 raft_consensus.cc:385] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:30.961910 11553 raft_consensus.cc:740] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:30.961829 11641 raft_consensus.cc:515] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.962978 11553 consensus_queue.cc:260] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.971585 11554 raft_consensus.cc:359] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:30.974946 11554 raft_consensus.cc:385] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:30.975354 11554 raft_consensus.cc:740] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:30.969038 11656 consensus_queue.cc:237] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:30.976550 11554 consensus_queue.cc:260] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:30.977262 11289 consensus_peers.cc:597] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:30.980862 11554 ts_tablet_manager.cc:1434] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.037s user 0.003s sys 0.005s
I20260809 01:45:30.982323 11553 ts_tablet_manager.cc:1434] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.025s user 0.007s sys 0.000s
I20260809 01:45:30.983245 11553 tablet_bootstrap.cc:492] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:30.990176 11551 tablet_bootstrap.cc:492] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:30.990813 11551 ts_tablet_manager.cc:1403] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.448s user 0.022s sys 0.051s
I20260809 01:45:30.994475 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_317180e171034a9db048d05d2f69585b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:30.996981 11641 leader_election.cc:290] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
W20260809 01:45:30.999794 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_317180e171034a9db048d05d2f69585b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:31.006052 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_71d1332c30c4432791d1a84d52510063" } new_table_name: "default.Table_5f4d617a86c2487fa9f64cb58f14de3c" modify_external_catalogs: true
I20260809 01:45:31.006013 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e155d16b1d594a789f539390a44bc39f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:31.006852 10905 raft_consensus.cc:2468] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:31.014235 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_9590b2028c934cbb91c44edf337bb871"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:31.015324 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b7459ad0974cb9948d647459b22260" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:31.015949 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ac88034c7a4dbb82a39ac7731c37e5" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:31.016279 11395 raft_consensus.cc:2468] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 0.
I20260809 01:45:31.016619 10910 raft_consensus.cc:3060] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:31.016732 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9590b2028c934cbb91c44edf337bb871 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:31.015357 11289 leader_election.cc:304] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:31.019275 11601 raft_consensus.cc:2804] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:31.019896 11601 raft_consensus.cc:493] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:31.020390 11601 raft_consensus.cc:3060] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:31.029084 10805 leader_election.cc:304] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:31.030539 11641 raft_consensus.cc:2804] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:31.030992 11641 raft_consensus.cc:493] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:31.031567 11641 raft_consensus.cc:3060] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:31.031057 11601 raft_consensus.cc:515] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.037238 11601 leader_election.cc:290] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:31.042143 11595 ts_tablet_manager.cc:1929] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:31.042689 11595 log.cc:1199] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/f3c0c5eaa1f64c0298226901de71e7f9
I20260809 01:45:31.043066 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e155d16b1d594a789f539390a44bc39f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:31.044042 10905 raft_consensus.cc:3060] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:31.053860 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_a0d298080721403d9305bc761dda44a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:31.056608 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a0d298080721403d9305bc761dda44a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:31.071391 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b7459ad0974cb9948d647459b22260" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:31.073815 10804 leader_election.cc:343] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:31.074283 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 20) took 1692 ms. Trace:
I20260809 01:45:31.074648 11565 rpcz_store.cc:276] 0809 01:45:29.382092 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:29.382375 (+ 283us) service_pool.cc:224] Handling call
0809 01:45:30.204094 (+821719us) ts_tablet_manager.cc:1151] Deleting tablet 5b2b6684972a477e83b19d977f0c1e7c
0809 01:45:30.204179 (+ 85us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:30.470974 (+266795us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:30.891172 (+420198us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:31.074221 (+183049us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":730182}
I20260809 01:45:31.075029 10905 raft_consensus.cc:2468] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:31.076241 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 24) took 1001 ms. Trace:
I20260809 01:45:30.993456 11062 leader_election.cc:304] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:31.076722 11289 leader_election.cc:304] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
W20260809 01:45:31.077689 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 110319us
I20260809 01:45:31.077634 11642 raft_consensus.cc:2804] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:31.078351 11642 raft_consensus.cc:697] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:31.079370 11551 raft_consensus.cc:359] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.079957 11551 raft_consensus.cc:385] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:31.080219 11551 raft_consensus.cc:740] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:31.061111 11589 raft_consensus.cc:493] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:31.082849 11589 raft_consensus.cc:515] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.079766 11642 consensus_queue.cc:237] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.086033 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e155d16b1d594a789f539390a44bc39f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:31.086767 11158 raft_consensus.cc:2468] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:31.087008 11554 tablet_bootstrap.cc:492] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
W20260809 01:45:31.086591 10804 consensus_peers.cc:597] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5b2b6684972a477e83b19d977f0c1e7c. This is attempt 1: this message will repeat every 5th retry.
W20260809 01:45:31.090140 10805 consensus_peers.cc:597] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5b2b6684972a477e83b19d977f0c1e7c. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:31.092067 10910 raft_consensus.cc:2468] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:31.093549 11553 tablet_bootstrap.cc:654] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:31.093405 11283 leader_election.cc:343] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: 64ac88034c7a4dbb82a39ac7731c37e5
I20260809 01:45:31.093701 11595 ts_tablet_manager.cc:1950] T f3c0c5eaa1f64c0298226901de71e7f9 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:31.097491 11595 tablet_replica.cc:333] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:31.097709 11601 raft_consensus.cc:2804] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:31.098459 11601 raft_consensus.cc:697] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:31.099069 11595 raft_consensus.cc:2243] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:31.099783 11595 raft_consensus.cc:2272] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:31.099375 11601 consensus_queue.cc:237] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.102990 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet f3c0c5eaa1f64c0298226901de71e7f9 (table default.Table_6149470a609f459c951f42acf3332a0d [id=71f0e1b66e0d466ba1297d4ce6c85ad9]) successfully deleted
I20260809 01:45:31.103725 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 5b2b6684972a477e83b19d977f0c1e7c (table default.Table_aed6870dac374d5d9ecc995f90b09201 [id=1fc93322377e409b961a543dcd0d56f5]) successfully deleted
I20260809 01:45:31.103971 11564 tablet_replica.cc:333] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:31.084995 11641 raft_consensus.cc:515] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.105345 11564 raft_consensus.cc:2243] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:31.107460 11554 tablet_bootstrap.cc:654] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:31.107703 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b7459ad0974cb9948d647459b22260" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:31.108510 11395 raft_consensus.cc:3060] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:31.111050 11595 ts_tablet_manager.cc:1916] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:31.080700 11551 consensus_queue.cc:260] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.116814 11551 ts_tablet_manager.cc:1434] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.126s user 0.004s sys 0.002s
I20260809 01:45:31.118144 11564 pending_rounds.cc:70] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316: Trying to abort 2 pending ops.
I20260809 01:45:31.117573 11565 rpcz_store.cc:276] 0809 01:45:30.074680 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:30.074866 (+ 186us) service_pool.cc:224] Handling call
0809 01:45:31.075878 (+1001012us) ts_tablet_manager.cc:1151] Deleting tablet 2674c9dbf9e44d63b41427f2fda4e311
0809 01:45:31.075929 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:31.076184 (+ 255us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":938071}
I20260809 01:45:31.119204 11565 tablet_replica.cc:333] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:31.118682 11564 pending_rounds.cc:77] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316438756491104256 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "f3c0c5eaa1f64c0298226901de71e7f9" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_6149470a609f459c951f42acf3332a0d" dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" new_extra_config { } }
I20260809 01:45:31.120114 11565 raft_consensus.cc:2243] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:31.120388 11395 raft_consensus.cc:2468] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 1.
I20260809 01:45:31.120645 11565 raft_consensus.cc:2272] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:31.120965 11564 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53476: Aborted: Op aborted
I20260809 01:45:31.121968 10805 leader_election.cc:304] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:31.123131 11624 raft_consensus.cc:2804] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:31.123466 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e155d16b1d594a789f539390a44bc39f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:31.124238 11160 raft_consensus.cc:3060] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:31.127478 11641 leader_election.cc:290] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:31.130810 11553 tablet_bootstrap.cc:492] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:31.131273 11553 ts_tablet_manager.cc:1403] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.148s user 0.009s sys 0.009s
I20260809 01:45:31.134064 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 5b2b6684972a477e83b19d977f0c1e7c (table default.Table_aed6870dac374d5d9ecc995f90b09201 [id=1fc93322377e409b961a543dcd0d56f5]) successfully deleted
I20260809 01:45:31.138299 11589 leader_election.cc:290] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:31.138365 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e155d16b1d594a789f539390a44bc39f" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:31.139649 10905 raft_consensus.cc:2393] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33b5652538e64ed58a09993d3188f8d7 in current term 1: Already voted for candidate 4f60c9ac9414403d9ea5a97b103c2316 in this term.
I20260809 01:45:31.143743 11564 pending_rounds.cc:77] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438751567904768 op_type: NO_OP noop_request { }
I20260809 01:45:31.144610 11553 raft_consensus.cc:359] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.145540 11553 raft_consensus.cc:385] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:31.145810 11553 raft_consensus.cc:740] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:31.146173 11564 raft_consensus.cc:2889] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:31.146580 11553 consensus_queue.cc:260] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:31.147182 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 2674c9dbf9e44d63b41427f2fda4e311 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2674c9dbf9e44d63b41427f2fda4e311 already in progress: creating tablet
I20260809 01:45:31.147861 11624 raft_consensus.cc:697] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Becoming Leader. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Running, Role: LEADER
I20260809 01:45:31.149315 11624 consensus_queue.cc:237] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.148101 11564 raft_consensus.cc:2272] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:31.153574 11553 ts_tablet_manager.cc:1434] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.022s user 0.006s sys 0.004s
I20260809 01:45:31.154453 11553 tablet_bootstrap.cc:492] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:31.156327 11564 ts_tablet_manager.cc:1916] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:31.167413 11553 tablet_bootstrap.cc:654] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:31.177065 11551 tablet_bootstrap.cc:492] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:31.181869 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b7459ad0974cb9948d647459b22260" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:31.159389 11289 leader_election.cc:304] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:31.184811 11655 raft_consensus.cc:2804] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
W20260809 01:45:31.189841 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 96293.1us
W20260809 01:45:31.193137 10804 leader_election.cc:343] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:31.168807 11554 tablet_bootstrap.cc:492] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:31.195079 11554 ts_tablet_manager.cc:1403] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.108s user 0.007s sys 0.009s
I20260809 01:45:31.195230 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e155d16b1d594a789f539390a44bc39f" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:31.198084 11062 leader_election.cc:304] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:31.199165 10596 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ead75d766d0346f6823074d90ba8054e consensus state has no leader
I20260809 01:45:31.203081 11124 tablet_service.cc:1511] Processing CreateTablet for tablet ead75d766d0346f6823074d90ba8054e (DEFAULT_TABLE table=default.Table_be8856c44b864dd6be36f611dfaf4769 [id=028331846d434fc1beff5b11e2129233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:31.204392 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ead75d766d0346f6823074d90ba8054e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:31.216207 11369 tablet_service.cc:1511] Processing CreateTablet for tablet ead75d766d0346f6823074d90ba8054e (DEFAULT_TABLE table=default.Table_be8856c44b864dd6be36f611dfaf4769 [id=028331846d434fc1beff5b11e2129233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:31.212641 10599 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet dfb4ad36ca954c7c99407612c0f30711 (table default.Table_a3341f14abab492293b1beadb947e74b [id=7831718cddee4b5b9c42cec19ff7d074]). Expected version 2 got 0
I20260809 01:45:31.223238 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ead75d766d0346f6823074d90ba8054e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:31.232838 11595 ts_tablet_manager.cc:1929] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:31.235055 11595 log.cc:1199] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/825c568f98014fbf92ade7bd99763581
I20260809 01:45:31.244563 11595 ts_tablet_manager.cc:1950] T 825c568f98014fbf92ade7bd99763581 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:31.249192 11595 tablet_replica.cc:333] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:31.249253 10588 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 72b4850de7d740d794283ca93152fdc1 (table default.Table_8f1bad3be6ed41f086fd8a9cf367555a [id=46b096a453084568935bc0d5855cb0f7]). Expected version 2 got 0
I20260809 01:45:31.259625 11595 raft_consensus.cc:2243] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:31.260820 11595 pending_rounds.cc:70] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8: Trying to abort 1 pending ops.
I20260809 01:45:31.266448 11565 ts_tablet_manager.cc:1916] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:31.261227 11595 pending_rounds.cc:77] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438753976676352 op_type: NO_OP noop_request { }
I20260809 01:45:31.270292 11595 raft_consensus.cc:2889] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:31.270709 11595 raft_consensus.cc:2272] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:31.274603 11595 ts_tablet_manager.cc:1916] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:31.277221 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 825c568f98014fbf92ade7bd99763581 (table default.Table_006a6ebb9836470396c69b4b78abc954 [id=366d143256e84961b8b8ae2bc08452d0]) successfully deleted
W20260809 01:45:31.278719 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 9bdd0ebb4ff348db986bf29913d430ff because the tablet was not found. No further retry: Not found: Tablet not found: 9bdd0ebb4ff348db986bf29913d430ff
I20260809 01:45:31.281061 10873 tablet_service.cc:1511] Processing CreateTablet for tablet ead75d766d0346f6823074d90ba8054e (DEFAULT_TABLE table=default.Table_be8856c44b864dd6be36f611dfaf4769 [id=028331846d434fc1beff5b11e2129233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:31.282811 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ead75d766d0346f6823074d90ba8054e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:31.301878 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5bb5e2ec60c24f8396daed992ca314ac consensus state has no leader
I20260809 01:45:31.334276 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 64ac88034c7a4dbb82a39ac7731c37e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:31.342486 10586 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 43d84f310d8e4df7b8b2d602289025ae
I20260809 01:45:31.352751 11595 ts_tablet_manager.cc:1929] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:31.353497 11595 log.cc:1199] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/5b2b6684972a477e83b19d977f0c1e7c
I20260809 01:45:31.354971 11595 ts_tablet_manager.cc:1950] T 5b2b6684972a477e83b19d977f0c1e7c P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:31.355394 10592 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet af8056e8875c42948923e1d76e49d6d0
I20260809 01:45:31.358227 11595 tablet_replica.cc:333] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:31.359685 11595 raft_consensus.cc:2243] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:31.360504 11595 raft_consensus.cc:2272] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:31.363559 11595 ts_tablet_manager.cc:1916] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:31.363976 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 43d84f310d8e4df7b8b2d602289025ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:31.365609 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet af8056e8875c42948923e1d76e49d6d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:31.367475 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 5b2b6684972a477e83b19d977f0c1e7c (table default.Table_aed6870dac374d5d9ecc995f90b09201 [id=1fc93322377e409b961a543dcd0d56f5]) successfully deleted
I20260809 01:45:31.371850 11126 tablet_service.cc:1558] Processing DeleteTablet for tablet af8056e8875c42948923e1d76e49d6d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:31.372676 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 98bee541fc1345fa83ffebeba05d8c8e (DEFAULT_TABLE table=default.Table_e212a7cfad494f2d97ac5eada6ac15f5 [id=d2c591183b4646ca88b53698200a62c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:31.374043 11123 tablet_service.cc:1558] Processing DeleteTablet for tablet 43d84f310d8e4df7b8b2d602289025ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:31.389961 11595 ts_tablet_manager.cc:1929] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:31.390506 11595 log.cc:1199] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/64ac88034c7a4dbb82a39ac7731c37e5
I20260809 01:45:31.391268 11595 ts_tablet_manager.cc:1950] T 64ac88034c7a4dbb82a39ac7731c37e5 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:31.393981 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98bee541fc1345fa83ffebeba05d8c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:31.401415 10873 tablet_service.cc:1511] Processing CreateTablet for tablet 98bee541fc1345fa83ffebeba05d8c8e (DEFAULT_TABLE table=default.Table_e212a7cfad494f2d97ac5eada6ac15f5 [id=d2c591183b4646ca88b53698200a62c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:31.403388 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98bee541fc1345fa83ffebeba05d8c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:31.404023 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 43d84f310d8e4df7b8b2d602289025ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:31.405972 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet af8056e8875c42948923e1d76e49d6d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:31.407395 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 64ac88034c7a4dbb82a39ac7731c37e5 (table default.Table_aaad8afaf8134c808577f0b370975c6c [id=7ff740ce8d4f4b5995b4d5ef52cafada]) successfully deleted
I20260809 01:45:31.435005 10592 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_7033d361a8614225b9dcf10aa47a5bd9" } new_table_name: "default.Table_ca43ce2300964cea8a0b69061d48fed0" modify_external_catalogs: true
I20260809 01:45:31.441649 11655 raft_consensus.cc:697] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:31.443732 11655 consensus_queue.cc:237] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:31.460772 11160 raft_consensus.cc:2468] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
W20260809 01:45:31.478348 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 43d84f310d8e4df7b8b2d602289025ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43d84f310d8e4df7b8b2d602289025ae already in progress: creating tablet
I20260809 01:45:31.480633 11682 raft_consensus.cc:493] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:31.481127 11682 raft_consensus.cc:515] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.485967 11682 leader_election.cc:290] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
W20260809 01:45:31.493887 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet af8056e8875c42948923e1d76e49d6d0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet af8056e8875c42948923e1d76e49d6d0 already in progress: creating tablet
I20260809 01:45:31.519860 11553 tablet_bootstrap.cc:492] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:31.519681 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_d5bd8c94a186431ba5f5b8fa23d75b8b" } new_table_name: "default.Table_d0ea8941c02b422a945c246593ba62c3" modify_external_catalogs: true
I20260809 01:45:31.520375 11553 ts_tablet_manager.cc:1403] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.366s user 0.059s sys 0.137s
I20260809 01:45:31.450917 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 98bee541fc1345fa83ffebeba05d8c8e (DEFAULT_TABLE table=default.Table_e212a7cfad494f2d97ac5eada6ac15f5 [id=d2c591183b4646ca88b53698200a62c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:31.527086 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98bee541fc1345fa83ffebeba05d8c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:31.532158 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_d96a5c9fd7204aaba827a8b16b1c9e62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:31.525336 11553 raft_consensus.cc:359] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.534143 11553 raft_consensus.cc:385] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:31.534422 11553 raft_consensus.cc:740] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:31.535195 11553 consensus_queue.cc:260] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:31.543536 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d96a5c9fd7204aaba827a8b16b1c9e62 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:31.547565 10873 tablet_service.cc:1511] Processing CreateTablet for tablet 5bb5e2ec60c24f8396daed992ca314ac (DEFAULT_TABLE table=default.Table_5f4d617a86c2487fa9f64cb58f14de3c [id=f64daa5a2706436081e9166a5f3a0cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:31.547740 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 5bb5e2ec60c24f8396daed992ca314ac (DEFAULT_TABLE table=default.Table_5f4d617a86c2487fa9f64cb58f14de3c [id=f64daa5a2706436081e9166a5f3a0cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:31.549742 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bb5e2ec60c24f8396daed992ca314ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:31.552582 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bb5e2ec60c24f8396daed992ca314ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:31.493818 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b4850de7d740d794283ca93152fdc1" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:31.561728 11589 raft_consensus.cc:2749] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:31.562801 11553 ts_tablet_manager.cc:1434] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.017s user 0.003s sys 0.003s
I20260809 01:45:31.565671 11553 tablet_bootstrap.cc:492] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:31.537030 11551 tablet_bootstrap.cc:654] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:31.540484 11123 tablet_service.cc:1511] Processing CreateTablet for tablet 5bb5e2ec60c24f8396daed992ca314ac (DEFAULT_TABLE table=default.Table_5f4d617a86c2487fa9f64cb58f14de3c [id=f64daa5a2706436081e9166a5f3a0cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:31.570864 11123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bb5e2ec60c24f8396daed992ca314ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:31.581465 11126 tablet_service.cc:1511] Processing CreateTablet for tablet 0da8d25290814e66b99c2cc1d5f8b0cf (DEFAULT_TABLE table=default.Table_84e4a5312bbb43c9bd562d0f742537e2 [id=5cc4859efd27438a8570acaafe1e6e78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:31.583714 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0da8d25290814e66b99c2cc1d5f8b0cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:31.592991 11564 ts_tablet_manager.cc:1929] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:45:31.600553 11553 tablet_bootstrap.cc:654] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:31.609015 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 34b103e2d3854eb4bf8e5cd649cbe129 (DEFAULT_TABLE table=default.Table_175d0b46158d49a88aac3833ba31533a [id=cf3f859364164b9085d525891266a825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:31.615860 11564 log.cc:1199] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/f3c0c5eaa1f64c0298226901de71e7f9
I20260809 01:45:31.617453 11564 ts_tablet_manager.cc:1950] T f3c0c5eaa1f64c0298226901de71e7f9 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:31.621901 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 0da8d25290814e66b99c2cc1d5f8b0cf (DEFAULT_TABLE table=default.Table_84e4a5312bbb43c9bd562d0f742537e2 [id=5cc4859efd27438a8570acaafe1e6e78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:31.623698 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0da8d25290814e66b99c2cc1d5f8b0cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:31.631580 11564 tablet_replica.cc:333] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:31.632982 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34b103e2d3854eb4bf8e5cd649cbe129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:31.637943 11564 raft_consensus.cc:2243] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:31.638724 11564 raft_consensus.cc:2272] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:31.640146 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 0da8d25290814e66b99c2cc1d5f8b0cf (DEFAULT_TABLE table=default.Table_84e4a5312bbb43c9bd562d0f742537e2 [id=5cc4859efd27438a8570acaafe1e6e78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:31.641932 11564 ts_tablet_manager.cc:1916] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:31.642092 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0da8d25290814e66b99c2cc1d5f8b0cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:31.649960 11551 tablet_bootstrap.cc:492] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:31.650480 11551 ts_tablet_manager.cc:1403] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.533s user 0.016s sys 0.005s
I20260809 01:45:31.652637 11553 tablet_bootstrap.cc:492] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:31.653052 11553 ts_tablet_manager.cc:1403] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.088s user 0.003s sys 0.015s
I20260809 01:45:31.653123 11551 raft_consensus.cc:359] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.653733 11551 raft_consensus.cc:385] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:31.653983 11551 raft_consensus.cc:740] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:31.654604 11551 consensus_queue.cc:260] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.657292 11553 raft_consensus.cc:359] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.658192 11553 raft_consensus.cc:385] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:31.658509 11553 raft_consensus.cc:740] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:31.659343 11553 consensus_queue.cc:260] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.533747 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_d29473576bc0470bbea9f4ef1fb00e8b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:31.667457 11551 ts_tablet_manager.cc:1434] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260809 01:45:31.554026 10598 catalog_manager.cc:5719] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65). New cstate: current_term: 1 leader_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:31.668469 11551 tablet_bootstrap.cc:492] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:31.669252 10598 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 72b4850de7d740d794283ca93152fdc1 (table default.Table_8f1bad3be6ed41f086fd8a9cf367555a [id=46b096a453084568935bc0d5855cb0f7]). Expected version 2 got 0
W20260809 01:45:31.670317 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d29473576bc0470bbea9f4ef1fb00e8b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:31.671782 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet f3c0c5eaa1f64c0298226901de71e7f9 (table default.Table_6149470a609f459c951f42acf3332a0d [id=71f0e1b66e0d466ba1297d4ce6c85ad9]) successfully deleted
I20260809 01:45:31.561412 10589 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_a3341f14abab492293b1beadb947e74b" } modify_external_catalogs: true
I20260809 01:45:31.564327 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_fabd51c93e3347cdb50fb9a28d32ff29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:31.676133 11553 ts_tablet_manager.cc:1434] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.023s user 0.008s sys 0.000s
W20260809 01:45:31.675909 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fabd51c93e3347cdb50fb9a28d32ff29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:31.678506 11369 tablet_service.cc:1511] Processing CreateTablet for tablet 34b103e2d3854eb4bf8e5cd649cbe129 (DEFAULT_TABLE table=default.Table_175d0b46158d49a88aac3833ba31533a [id=cf3f859364164b9085d525891266a825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:31.680742 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34b103e2d3854eb4bf8e5cd649cbe129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:31.591157 10596 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_175d0b46158d49a88aac3833ba31533a" } modify_external_catalogs: true
I20260809 01:45:31.684912 10867 tablet_service.cc:1511] Processing CreateTablet for tablet 34b103e2d3854eb4bf8e5cd649cbe129 (DEFAULT_TABLE table=default.Table_175d0b46158d49a88aac3833ba31533a [id=cf3f859364164b9085d525891266a825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:31.685631 11624 raft_consensus.cc:493] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:31.687072 10867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34b103e2d3854eb4bf8e5cd649cbe129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:31.686559 11624 raft_consensus.cc:515] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.691915 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c4892ed69143cfb9659a6f4a9d9e55" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:31.692121 11624 leader_election.cc:290] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:31.693452 11398 raft_consensus.cc:2393] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5117038a52a45b0aea3d30ca9cfcbe8 in current term 1: Already voted for candidate 33b5652538e64ed58a09993d3188f8d7 in this term.
I20260809 01:45:31.700577 11565 ts_tablet_manager.cc:1929] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:31.676934 11553 tablet_bootstrap.cc:492] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
W20260809 01:45:31.710039 10804 leader_election.cc:343] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:31.455374 11554 raft_consensus.cc:359] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.485672 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b4850de7d740d794283ca93152fdc1" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:31.713609 11395 raft_consensus.cc:2468] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 0.
I20260809 01:45:31.715364 11554 raft_consensus.cc:385] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:31.715770 11554 raft_consensus.cc:740] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:31.715828 10805 leader_election.cc:304] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:31.718010 11624 raft_consensus.cc:2804] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:31.718608 11624 raft_consensus.cc:493] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:31.719136 11624 raft_consensus.cc:3060] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:31.731551 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c4892ed69143cfb9659a6f4a9d9e55" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:31.733667 10589 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_a3341f14abab492293b1beadb947e74b" } modify_external_catalogs: true
W20260809 01:45:31.734896 10592 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ead75d766d0346f6823074d90ba8054e consensus state has no leader
I20260809 01:45:31.743521 10596 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_175d0b46158d49a88aac3833ba31533a" } modify_external_catalogs: true
W20260809 01:45:31.748744 11062 consensus_peers.cc:597] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:31.753461 10804 leader_election.cc:304] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:31.736047 11565 log.cc:1199] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/7e79e27a6c44498b99f363dd654fbb7e
I20260809 01:45:31.758647 11565 ts_tablet_manager.cc:1950] T 7e79e27a6c44498b99f363dd654fbb7e P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:31.761354 11682 raft_consensus.cc:3060] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:31.762409 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 25) took 1677 ms. Trace:
I20260809 01:45:31.763012 11565 rpcz_store.cc:276] 0809 01:45:30.085134 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:30.085290 (+ 156us) service_pool.cc:224] Handling call
0809 01:45:31.118943 (+1033653us) ts_tablet_manager.cc:1151] Deleting tablet 7e79e27a6c44498b99f363dd654fbb7e
0809 01:45:31.119018 (+ 75us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:31.608387 (+489369us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:31.700396 (+ 92009us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:31.762312 (+ 61916us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":980771}
I20260809 01:45:31.765506 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 7e79e27a6c44498b99f363dd654fbb7e (table default.Table_b9f38ba904ef48559bddb776d4f3a4ff [id=1bdacb31d36c44f58ce3b2264fd8cfcc]) successfully deleted
W20260809 01:45:31.772152 10576 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet ead75d766d0346f6823074d90ba8054e consensus state has no leader
I20260809 01:45:31.775976 11551 tablet_bootstrap.cc:654] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:31.777734 11682 raft_consensus.cc:2749] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:31.782596 11553 tablet_bootstrap.cc:654] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:31.778636 11624 raft_consensus.cc:515] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.716881 11554 consensus_queue.cc:260] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.803684 11554 ts_tablet_manager.cc:1434] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.601s user 0.008s sys 0.000s
W20260809 01:45:31.804090 11554 ts_tablet_manager.cc:1467] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316: Tablet startup took 601ms
I20260809 01:45:31.809296 11589 consensus_queue.cc:1048] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:31.830860 11624 leader_election.cc:290] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:31.834354 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b4850de7d740d794283ca93152fdc1" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:31.835278 11398 raft_consensus.cc:3060] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:31.838757 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 12fb852b8ea8478cb160a265459865c4 has no consensus state
I20260809 01:45:31.839543 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b4850de7d740d794283ca93152fdc1" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:31.840600 11565 tablet_replica.cc:333] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
W20260809 01:45:31.845216 10804 leader_election.cc:343] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:31.847388 11589 consensus_queue.cc:1048] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:31.857180 11398 raft_consensus.cc:2468] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 1.
I20260809 01:45:31.858207 11565 raft_consensus.cc:2243] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:31.862411 11565 pending_rounds.cc:70] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7: Trying to abort 1 pending ops.
I20260809 01:45:31.864614 10805 leader_election.cc:304] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:31.864950 11589 consensus_queue.cc:1048] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:31.875921 11624 raft_consensus.cc:2804] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:31.876785 11624 raft_consensus.cc:697] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Becoming Leader. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Running, Role: LEADER
I20260809 01:45:31.888325 11565 pending_rounds.cc:77] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438751567904768 op_type: NO_OP noop_request { }
I20260809 01:45:31.888931 11565 raft_consensus.cc:2889] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:31.889633 11565 raft_consensus.cc:2272] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:31.893575 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet a8c4892ed69143cfb9659a6f4a9d9e55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:31.894737 11595 tablet_replica.cc:333] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:31.897003 11624 consensus_queue.cc:237] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.903067 11655 raft_consensus.cc:493] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:31.903913 11655 raft_consensus.cc:515] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:31.804443 11554 ts_tablet_manager.cc:1469] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316: Trace:
0809 01:45:29.329383 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:29.330396 (+ 1013us) service_pool.cc:224] Handling call
0809 01:45:29.332932 (+ 2536us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:29.333072 (+ 140us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:29.491588 (+158516us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:29.572015 (+ 80427us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:29.572428 (+ 413us) inbound_call.cc:177] Queueing success response
0809 01:45:31.086707 (+1514279us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0809 01:45:31.203021 (+116314us) ts_tablet_manager.cc:1435] Starting tablet replica
0809 01:45:31.204866 (+ 1845us) tablet_replica.cc:261] Starting instrumentation
0809 01:45:31.453622 (+248756us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":177,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":345,"tablet-open.queue_time_us":1514349}
I20260809 01:45:31.906790 11554 tablet_bootstrap.cc:492] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:31.908968 11551 tablet_bootstrap.cc:492] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:31.909426 11551 ts_tablet_manager.cc:1403] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.241s user 0.014s sys 0.004s
I20260809 01:45:31.910379 11595 raft_consensus.cc:2243] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:31.911446 11595 pending_rounds.cc:70] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8: Trying to abort 1 pending ops.
I20260809 01:45:31.911749 11595 pending_rounds.cc:77] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438757749927936 op_type: NO_OP noop_request { }
I20260809 01:45:31.912132 11595 raft_consensus.cc:2889] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:31.912544 11595 raft_consensus.cc:2272] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:31.915824 11655 leader_election.cc:290] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
W20260809 01:45:31.917443 11063 consensus_peers.cc:597] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:31.919212 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e79e27a6c44498b99f363dd654fbb7e" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:31.919965 10908 raft_consensus.cc:2468] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
W20260809 01:45:31.917568 11283 leader_election.cc:343] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: 7e79e27a6c44498b99f363dd654fbb7e
I20260809 01:45:31.927882 11289 leader_election.cc:304] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:31.929329 11564 ts_tablet_manager.cc:1929] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:31.932717 11564 log.cc:1199] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/825c568f98014fbf92ade7bd99763581
I20260809 01:45:31.929754 11655 raft_consensus.cc:2804] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:31.940775 11655 raft_consensus.cc:493] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:31.941464 11655 raft_consensus.cc:3060] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:31.946516 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_9081f03ef3eb4bb49093311263bb3976" } new_table_name: "default.Table_57234dd818414b7397d1ac621bdb06a7" modify_external_catalogs: true
I20260809 01:45:31.946986 11565 ts_tablet_manager.cc:1916] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:31.947111 11656 consensus_queue.cc:1048] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:31.958189 11595 ts_tablet_manager.cc:1916] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:31.963897 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_03cfb94f08614dc5bcdaf9eb182adc1b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:31.967557 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03cfb94f08614dc5bcdaf9eb182adc1b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:31.971446 11564 ts_tablet_manager.cc:1950] T 825c568f98014fbf92ade7bd99763581 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:31.973443 11553 tablet_bootstrap.cc:492] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:31.974000 11553 ts_tablet_manager.cc:1403] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.297s user 0.007s sys 0.021s
I20260809 01:45:31.981621 11553 raft_consensus.cc:359] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:31.982530 11553 raft_consensus.cc:385] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:31.982880 11553 raft_consensus.cc:740] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:31.956123 11551 raft_consensus.cc:359] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.000046 11553 consensus_queue.cc:260] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.002563 11553 ts_tablet_manager.cc:1434] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.028s user 0.004s sys 0.005s
I20260809 01:45:32.003595 11553 tablet_bootstrap.cc:492] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:32.024793 11655 raft_consensus.cc:515] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.033496 11595 ts_tablet_manager.cc:1929] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:32.036877 11564 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 32) took 1336 ms. Trace:
I20260809 01:45:32.044453 11655 leader_election.cc:290] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:32.037509 11564 rpcz_store.cc:276] 0809 01:45:30.700914 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:30.701090 (+ 176us) service_pool.cc:224] Handling call
0809 01:45:31.631320 (+930230us) ts_tablet_manager.cc:1151] Deleting tablet 825c568f98014fbf92ade7bd99763581
0809 01:45:31.631400 (+ 80us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:31.770081 (+138681us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:31.929167 (+159086us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:32.036767 (+107600us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":801524}
I20260809 01:45:32.047428 11564 tablet_replica.cc:333] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:32.047580 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 825c568f98014fbf92ade7bd99763581 (table default.Table_006a6ebb9836470396c69b4b78abc954 [id=366d143256e84961b8b8ae2bc08452d0]) successfully deleted
I20260809 01:45:32.048348 11564 raft_consensus.cc:2243] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:32.049114 11564 pending_rounds.cc:70] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316: Trying to abort 1 pending ops.
I20260809 01:45:32.049433 11564 pending_rounds.cc:77] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438759539621888 op_type: NO_OP noop_request { }
I20260809 01:45:32.046886 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e79e27a6c44498b99f363dd654fbb7e" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:32.057792 10905 raft_consensus.cc:3060] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:32.064345 10905 raft_consensus.cc:2468] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:32.066676 11595 log.cc:1199] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/a8c4892ed69143cfb9659a6f4a9d9e55
I20260809 01:45:32.067631 11595 ts_tablet_manager.cc:1950] T a8c4892ed69143cfb9659a6f4a9d9e55 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:32.070894 11553 tablet_bootstrap.cc:654] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:32.071487 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet a8c4892ed69143cfb9659a6f4a9d9e55 (table default.Table_13597d7e311b4ff59dc3782447360563 [id=dbdbdc9a4bd54ba2811644b79f56a451]) successfully deleted
I20260809 01:45:32.052435 11655 consensus_queue.cc:1048] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:32.065780 11289 leader_election.cc:304] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:32.075007 11723 raft_consensus.cc:2804] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:32.075356 11564 raft_consensus.cc:2889] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:32.075721 11564 raft_consensus.cc:2272] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:32.075565 10273 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
/tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.125:42063
--webserver_interface=127.10.8.125
--webserver_port=46803
--builtin_ntp_servers=127.10.8.84:40957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.8.126:42157,127.10.8.125:42063,127.10.8.124:33929
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260809 01:45:32.079077 11723 raft_consensus.cc:697] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:32.079980 11723 consensus_queue.cc:237] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.069484 11554 tablet_bootstrap.cc:654] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:32.089975 11564 ts_tablet_manager.cc:1916] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:32.092131 11624 raft_consensus.cc:493] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:32.092726 11624 raft_consensus.cc:515] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.096935 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f8824953c794450ba274b160d891fcb" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:32.098287 11624 leader_election.cc:290] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
W20260809 01:45:32.099113 10805 leader_election.cc:343] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:32.105559 11553 tablet_bootstrap.cc:492] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:32.105965 11553 ts_tablet_manager.cc:1403] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.103s user 0.011s sys 0.000s
I20260809 01:45:32.106997 11624 consensus_queue.cc:1048] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:32.108441 11553 raft_consensus.cc:359] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.109050 11553 raft_consensus.cc:385] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 01:45:32.117460 11063 consensus_peers.cc:597] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 825c568f98014fbf92ade7bd99763581. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:32.118618 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f8824953c794450ba274b160d891fcb" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:32.120529 11554 tablet_bootstrap.cc:492] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:32.120954 11554 ts_tablet_manager.cc:1403] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.214s user 0.008s sys 0.009s
W20260809 01:45:32.120853 10804 leader_election.cc:343] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:32.121299 10804 leader_election.cc:304] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:32.122445 11624 raft_consensus.cc:2749] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:32.125818 11283 leader_election.cc:343] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: 7e79e27a6c44498b99f363dd654fbb7e
I20260809 01:45:32.127182 11655 consensus_queue.cc:1048] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:31.997777 11551 raft_consensus.cc:385] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:32.131584 11551 raft_consensus.cc:740] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:32.132871 11551 consensus_queue.cc:260] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.109319 11553 raft_consensus.cc:740] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:32.146400 11554 raft_consensus.cc:359] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.147087 11554 raft_consensus.cc:385] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:32.147439 11554 raft_consensus.cc:740] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:32.147842 11159 raft_consensus.cc:3060] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:32.148268 11554 consensus_queue.cc:260] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.152230 11554 ts_tablet_manager.cc:1434] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.031s user 0.005s sys 0.002s
I20260809 01:45:32.154122 11554 tablet_bootstrap.cc:492] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:32.160992 11554 tablet_bootstrap.cc:654] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:32.146140 11551 ts_tablet_manager.cc:1434] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.236s user 0.009s sys 0.000s
I20260809 01:45:32.166635 11551 tablet_bootstrap.cc:492] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:32.196987 11624 consensus_queue.cc:1048] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:32.214498 11551 tablet_bootstrap.cc:654] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:32.240826 10591 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_317180e171034a9db048d05d2f69585b" } new_table_name: "default.Table_3239fc2ae3d348c39baafc0778a13a41" modify_external_catalogs: true
I20260809 01:45:32.265074 10599 master_service.cc:960] ReplaceTablet: received request to replace tablet dfb4ad36ca954c7c99407612c0f30711 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:32.266789 11553 consensus_queue.cc:260] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:32.277082 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 44ac6aecc93447f7935fbd656fd2a86b consensus state has no leader
I20260809 01:45:32.285861 11554 tablet_bootstrap.cc:492] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:32.286533 11554 ts_tablet_manager.cc:1403] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.133s user 0.030s sys 0.051s
I20260809 01:45:32.293601 11564 ts_tablet_manager.cc:1929] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:32.294032 11564 log.cc:1199] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/64ac88034c7a4dbb82a39ac7731c37e5
I20260809 01:45:32.294780 11564 ts_tablet_manager.cc:1950] T 64ac88034c7a4dbb82a39ac7731c37e5 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:32.295776 11564 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 33) took 1515 ms. Trace:
I20260809 01:45:32.296219 11564 rpcz_store.cc:276] 0809 01:45:30.780246 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:30.780400 (+ 154us) service_pool.cc:224] Handling call
0809 01:45:32.047239 (+1266839us) ts_tablet_manager.cc:1151] Deleting tablet 64ac88034c7a4dbb82a39ac7731c37e5
0809 01:45:32.047288 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:32.108969 (+ 61681us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:32.293451 (+184482us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:32.295713 (+ 2262us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1216668}
I20260809 01:45:32.291435 11554 raft_consensus.cc:359] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.297721 11554 raft_consensus.cc:385] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:32.298702 11554 raft_consensus.cc:740] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:32.298802 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 64ac88034c7a4dbb82a39ac7731c37e5 (table default.Table_aaad8afaf8134c808577f0b370975c6c [id=7ff740ce8d4f4b5995b4d5ef52cafada]) successfully deleted
I20260809 01:45:32.299425 11554 consensus_queue.cc:260] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.302165 11554 ts_tablet_manager.cc:1434] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.015s user 0.006s sys 0.003s
I20260809 01:45:32.307206 11553 ts_tablet_manager.cc:1434] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.201s user 0.006s sys 0.000s
I20260809 01:45:32.298448 11564 tablet_replica.cc:333] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
W20260809 01:45:32.310248 10591 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d0091542071441e5bad0c2dca819f37f has no consensus state
I20260809 01:45:32.309676 11564 raft_consensus.cc:2243] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:32.311811 11564 raft_consensus.cc:2272] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:32.312884 11554 tablet_bootstrap.cc:492] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:32.334415 11554 tablet_bootstrap.cc:654] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:32.338644 10592 catalog_manager.cc:5719] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67). New cstate: current_term: 1 leader_uuid: "4f60c9ac9414403d9ea5a97b103c2316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:32.344985 11564 ts_tablet_manager.cc:1916] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:32.346534 10870 tablet_service.cc:1511] Processing CreateTablet for tablet b870347b07b04de5b833ab4fd3a2475f (DEFAULT_TABLE table=default.Table_9590b2028c934cbb91c44edf337bb871 [id=a0eb60025e8c435e958f40f1ec5c366c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.348181 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b870347b07b04de5b833ab4fd3a2475f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.353044 11366 tablet_service.cc:1511] Processing CreateTablet for tablet b870347b07b04de5b833ab4fd3a2475f (DEFAULT_TABLE table=default.Table_9590b2028c934cbb91c44edf337bb871 [id=a0eb60025e8c435e958f40f1ec5c366c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.355026 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b870347b07b04de5b833ab4fd3a2475f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.374255 10867 tablet_service.cc:1511] Processing CreateTablet for tablet d0091542071441e5bad0c2dca819f37f (DEFAULT_TABLE table=default.Table_3239fc2ae3d348c39baafc0778a13a41 [id=266a0465cb4544eba6f3e9df1eacf724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.376216 10867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0091542071441e5bad0c2dca819f37f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.386191 10873 tablet_service.cc:1511] Processing CreateTablet for tablet 85d0a9d73d4841c393e7d2cef41fee89 (DEFAULT_TABLE table=default.Table_f860a62234ce42b8ae4c4ff3430df44c [id=ac1fb532e0824edaa1a3abc07758fbbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.388082 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d0a9d73d4841c393e7d2cef41fee89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.391644 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_f2379f27ebbb4266ab36dafd7d80f79f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:32.394097 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_d1d088df5e7d452ab5a8771a8d3679dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:32.396912 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1d088df5e7d452ab5a8771a8d3679dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:32.399197 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 2aac67e7819f4a70b1050b558868f27f (DEFAULT_TABLE table=default.Table_6df59952b936458893928a28040decda [id=04d4b772004b44e781c255cf271fab34]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:32.402911 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2379f27ebbb4266ab36dafd7d80f79f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:32.404305 10597 master_service.cc:960] ReplaceTablet: received request to replace tablet 61c83b8bcf54461bb24565e3a989100a from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:32.401475 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aac67e7819f4a70b1050b558868f27f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.404325 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_f6654879d18843fd84d44ea5f170dda7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:32.406112 11553 tablet_bootstrap.cc:492] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:32.415936 11126 tablet_service.cc:1511] Processing CreateTablet for tablet b870347b07b04de5b833ab4fd3a2475f (DEFAULT_TABLE table=default.Table_9590b2028c934cbb91c44edf337bb871 [id=a0eb60025e8c435e958f40f1ec5c366c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:32.418401 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f6654879d18843fd84d44ea5f170dda7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:32.418617 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 85d0a9d73d4841c393e7d2cef41fee89 (DEFAULT_TABLE table=default.Table_f860a62234ce42b8ae4c4ff3430df44c [id=ac1fb532e0824edaa1a3abc07758fbbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.420337 11129 tablet_service.cc:1511] Processing CreateTablet for tablet 2aac67e7819f4a70b1050b558868f27f (DEFAULT_TABLE table=default.Table_6df59952b936458893928a28040decda [id=04d4b772004b44e781c255cf271fab34]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:32.420732 10592 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.67:40463 (request call id 12) took 811 ms (client timeout 1000 ms). Trace:
I20260809 01:45:32.423033 11682 raft_consensus.cc:493] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:32.423076 11123 tablet_service.cc:1511] Processing CreateTablet for tablet d0091542071441e5bad0c2dca819f37f (DEFAULT_TABLE table=default.Table_3239fc2ae3d348c39baafc0778a13a41 [id=266a0465cb4544eba6f3e9df1eacf724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.423486 11682 raft_consensus.cc:515] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.424408 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b870347b07b04de5b833ab4fd3a2475f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.425422 11682 leader_election.cc:290] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:32.410331 10596 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 34b103e2d3854eb4bf8e5cd649cbe129
I20260809 01:45:32.428627 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet 34b103e2d3854eb4bf8e5cd649cbe129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:31 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:32.415741 10595 catalog_manager.cc:5719] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65). New cstate: current_term: 1 leader_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:32.429575 10595 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 72b4850de7d740d794283ca93152fdc1 (table default.Table_8f1bad3be6ed41f086fd8a9cf367555a [id=46b096a453084568935bc0d5855cb0f7]). Expected version 2 got 0
I20260809 01:45:32.430560 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet dfb4ad36ca954c7c99407612c0f30711
I20260809 01:45:32.430678 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2674c9dbf9e44d63b41427f2fda4e311" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:32.432572 10804 leader_election.cc:343] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:32.437958 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 34b103e2d3854eb4bf8e5cd649cbe129 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 34b103e2d3854eb4bf8e5cd649cbe129 already in progress: creating tablet
I20260809 01:45:32.433406 11123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0091542071441e5bad0c2dca819f37f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.453492 11125 tablet_service.cc:1558] Processing DeleteTablet for tablet 34b103e2d3854eb4bf8e5cd649cbe129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:31 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:32.455014 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet dfb4ad36ca954c7c99407612c0f30711 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:31 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:32.456344 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet a8c4892ed69143cfb9659a6f4a9d9e55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:32.457319 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aac67e7819f4a70b1050b558868f27f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.462002 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d0a9d73d4841c393e7d2cef41fee89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.466390 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet dfb4ad36ca954c7c99407612c0f30711 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:31 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:32.467170 11595 tablet_replica.cc:333] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:32.467811 11595 raft_consensus.cc:2243] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:32.468227 11595 raft_consensus.cc:2272] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:32.470681 11595 ts_tablet_manager.cc:1916] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:32.475468 10588 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.66:57751 (request call id 19) took 851 ms (client timeout 1000 ms). Trace:
W20260809 01:45:32.421118 10592 rpcz_store.cc:269] 0809 01:45:31.609784 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:31.737266 (+127482us) service_pool.cc:224] Handling call
0809 01:45:32.420665 (+683399us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:32.376125 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:32.376468 (+ 343us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:32.376490 (+ 22us) write_op.cc:435] Acquired schema lock
0809 01:45:32.376501 (+ 11us) tablet.cc:662] Decoding operations
0809 01:45:32.376722 (+ 221us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:32.376755 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:32.376777 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:32.376903 (+ 126us) tablet.cc:701] Row locks acquired
0809 01:45:32.376916 (+ 13us) write_op.cc:260] PREPARE: finished
0809 01:45:32.377016 (+ 100us) write_op.cc:270] Start()
0809 01:45:32.377087 (+ 71us) write_op.cc:276] Timestamp: P: 1786239932376963 usec, L: 0
0809 01:45:32.377101 (+ 14us) op_driver.cc:348] REPLICATION: starting
0809 01:45:32.377578 (+ 477us) log.cc:844] Serialized 526 byte log entry
0809 01:45:32.403077 (+ 25499us) op_driver.cc:464] REPLICATION: finished
0809 01:45:32.409273 (+ 6196us) write_op.cc:301] APPLY: starting
0809 01:45:32.409333 (+ 60us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:32.409421 (+ 88us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:32.409509 (+ 88us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:32.409684 (+ 175us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:32.409806 (+ 122us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:32.409850 (+ 44us) write_op.cc:312] APPLY: finished
0809 01:45:32.410035 (+ 185us) log.cc:844] Serialized 22 byte log entry
0809 01:45:32.410674 (+ 639us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:32.410739 (+ 65us) write_op.cc:454] Released schema lock
0809 01:45:32.410777 (+ 38us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":6008,"apply.run_cpu_time_us":1319,"apply.run_wall_time_us":53227,"num_ops":1,"prepare.queue_time_us":112,"prepare.run_cpu_time_us":1693,"prepare.run_wall_time_us":1842,"raft.queue_time_us":85,"raft.run_cpu_time_us":10,"raft.run_wall_time_us":11,"replication_time_us":25851}]]}
W20260809 01:45:32.475981 10588 rpcz_store.cc:269] 0809 01:45:31.589980 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:31.592482 (+ 2502us) service_pool.cc:224] Handling call
0809 01:45:32.441454 (+848972us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":4}
I20260809 01:45:32.491475 11682 raft_consensus.cc:493] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:32.492112 11682 raft_consensus.cc:515] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.497342 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af8056e8875c42948923e1d76e49d6d0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:32.499325 10804 leader_election.cc:343] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:32.499954 11682 leader_election.cc:290] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:32.502228 11565 ts_tablet_manager.cc:1929] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:32.502746 11565 log.cc:1199] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/f3c0c5eaa1f64c0298226901de71e7f9
I20260809 01:45:32.504279 11565 ts_tablet_manager.cc:1950] T f3c0c5eaa1f64c0298226901de71e7f9 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:32.520638 11655 raft_consensus.cc:493] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:32.521212 11655 raft_consensus.cc:515] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.524734 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2674c9dbf9e44d63b41427f2fda4e311" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:32.525400 11395 raft_consensus.cc:2468] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 0.
I20260809 01:45:32.526703 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af8056e8875c42948923e1d76e49d6d0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:32.527350 11398 raft_consensus.cc:2468] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 0.
I20260809 01:45:32.529487 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 61c83b8bcf54461bb24565e3a989100a
I20260809 01:45:32.532305 10597 catalog_manager.cc:6807] ReplaceTablet: tablet 61c83b8bcf54461bb24565e3a989100a deleted and replaced by tablet e5888017345c48c7bafe6f1c6e6945cc
I20260809 01:45:32.532426 11655 leader_election.cc:290] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:32.535502 10867 tablet_service.cc:1558] Processing DeleteTablet for tablet 61c83b8bcf54461bb24565e3a989100a with delete_type TABLET_DATA_DELETED (replaced by tablet e5888017345c48c7bafe6f1c6e6945cc) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:32.537806 10805 leader_election.cc:304] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:32.538888 10805 leader_election.cc:304] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:32.539986 11624 raft_consensus.cc:2804] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:32.540433 11624 raft_consensus.cc:493] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:32.540884 11624 raft_consensus.cc:3060] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:32.553194 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2674c9dbf9e44d63b41427f2fda4e311" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:32.561859 11564 ts_tablet_manager.cc:1929] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:32.563244 11564 log.cc:1199] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/a8c4892ed69143cfb9659a6f4a9d9e55
I20260809 01:45:32.566154 11564 ts_tablet_manager.cc:1950] T a8c4892ed69143cfb9659a6f4a9d9e55 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:32.575485 11682 raft_consensus.cc:2804] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:32.581760 11682 raft_consensus.cc:493] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:32.589694 11682 raft_consensus.cc:3060] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:32.621968 10867 tablet_service.cc:1511] Processing CreateTablet for tablet 12fb852b8ea8478cb160a265459865c4 (DEFAULT_TABLE table=default.Table_d0ea8941c02b422a945c246593ba62c3 [id=212edfae1bf849c3b46674ec191ee76d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.623469 10867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12fb852b8ea8478cb160a265459865c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.626113 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e79e27a6c44498b99f363dd654fbb7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:29 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:32.626463 11354 tablet_service.cc:1511] Processing CreateTablet for tablet d0091542071441e5bad0c2dca819f37f (DEFAULT_TABLE table=default.Table_3239fc2ae3d348c39baafc0778a13a41 [id=266a0465cb4544eba6f3e9df1eacf724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.627139 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet 64ac88034c7a4dbb82a39ac7731c37e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:32.628443 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet 34b103e2d3854eb4bf8e5cd649cbe129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:31 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:32.628954 11554 tablet_bootstrap.cc:492] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:32.629534 11554 ts_tablet_manager.cc:1403] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.317s user 0.017s sys 0.027s
I20260809 01:45:32.630065 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet dfb4ad36ca954c7c99407612c0f30711 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:31 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:32.631691 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet 61c83b8bcf54461bb24565e3a989100a with delete_type TABLET_DATA_DELETED (replaced by tablet e5888017345c48c7bafe6f1c6e6945cc) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:32.633059 11364 tablet_service.cc:1511] Processing CreateTablet for tablet 2aac67e7819f4a70b1050b558868f27f (DEFAULT_TABLE table=default.Table_6df59952b936458893928a28040decda [id=04d4b772004b44e781c255cf271fab34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.633216 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 85d0a9d73d4841c393e7d2cef41fee89 (DEFAULT_TABLE table=default.Table_f860a62234ce42b8ae4c4ff3430df44c [id=ac1fb532e0824edaa1a3abc07758fbbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.635852 11369 tablet_service.cc:1511] Processing CreateTablet for tablet 12fb852b8ea8478cb160a265459865c4 (DEFAULT_TABLE table=default.Table_d0ea8941c02b422a945c246593ba62c3 [id=212edfae1bf849c3b46674ec191ee76d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.411553 11553 tablet_bootstrap.cc:654] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:32.637497 11554 raft_consensus.cc:359] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.638311 11554 raft_consensus.cc:385] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:32.638628 11554 raft_consensus.cc:740] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:32.642068 10870 tablet_service.cc:1511] Processing CreateTablet for tablet c795f81d2bc248eeb39b64d6aaa0370c (DEFAULT_TABLE table=default.Table_4eef5dd402bc41168cd7b02c32e84835 [id=ff7d5da48c94470296b8c7154d68388c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.643770 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c795f81d2bc248eeb39b64d6aaa0370c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.651238 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0091542071441e5bad0c2dca819f37f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.654266 11364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aac67e7819f4a70b1050b558868f27f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.639447 11554 consensus_queue.cc:260] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.548651 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2674c9dbf9e44d63b41427f2fda4e311" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:32.550570 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet 61c83b8bcf54461bb24565e3a989100a with delete_type TABLET_DATA_DELETED (replaced by tablet e5888017345c48c7bafe6f1c6e6945cc) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:32.641430 11125 tablet_service.cc:1511] Processing CreateTablet for tablet 12fb852b8ea8478cb160a265459865c4 (DEFAULT_TABLE table=default.Table_d0ea8941c02b422a945c246593ba62c3 [id=212edfae1bf849c3b46674ec191ee76d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.654950 11120 tablet_service.cc:1511] Processing CreateTablet for tablet c795f81d2bc248eeb39b64d6aaa0370c (DEFAULT_TABLE table=default.Table_4eef5dd402bc41168cd7b02c32e84835 [id=ff7d5da48c94470296b8c7154d68388c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.669018 11120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c795f81d2bc248eeb39b64d6aaa0370c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.674358 11125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12fb852b8ea8478cb160a265459865c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.676599 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 737d7047a0c74a03b3203112987c0cb2 (DEFAULT_TABLE table=default.Table_25ec70c0a19d4c7dacff1c3a39e55e7e [id=ccd0ea07da604f859047f2eca85fc566]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.678184 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 737d7047a0c74a03b3203112987c0cb2 (DEFAULT_TABLE table=default.Table_25ec70c0a19d4c7dacff1c3a39e55e7e [id=ccd0ea07da604f859047f2eca85fc566]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.678777 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737d7047a0c74a03b3203112987c0cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.680017 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737d7047a0c74a03b3203112987c0cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.692620 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d0a9d73d4841c393e7d2cef41fee89. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:32.713166 10581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111) [suppressed 7 similar messages]
I20260809 01:45:32.716573 11553 tablet_bootstrap.cc:492] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:32.717064 11553 ts_tablet_manager.cc:1403] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.311s user 0.008s sys 0.003s
I20260809 01:45:32.721289 11553 raft_consensus.cc:359] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:32.721707 11283 leader_election.cc:343] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:32.721931 11553 raft_consensus.cc:385] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:32.722178 11553 raft_consensus.cc:740] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:32.722915 11553 consensus_queue.cc:260] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.723872 11554 ts_tablet_manager.cc:1434] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.094s user 0.007s sys 0.000s
I20260809 01:45:32.728991 11554 tablet_bootstrap.cc:492] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:32.738777 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12fb852b8ea8478cb160a265459865c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.742944 11361 tablet_service.cc:1511] Processing CreateTablet for tablet c795f81d2bc248eeb39b64d6aaa0370c (DEFAULT_TABLE table=default.Table_4eef5dd402bc41168cd7b02c32e84835 [id=ff7d5da48c94470296b8c7154d68388c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.744581 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c795f81d2bc248eeb39b64d6aaa0370c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.749744 11553 ts_tablet_manager.cc:1434] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.032s user 0.004s sys 0.004s
I20260809 01:45:32.750790 11553 tablet_bootstrap.cc:492] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
W20260809 01:45:32.772142 11744 log.cc:927] Time spent T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7: Append to log took a long time: real 0.173s user 0.001s sys 0.000s
I20260809 01:45:32.776423 11624 raft_consensus.cc:515] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.782398 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2674c9dbf9e44d63b41427f2fda4e311" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
W20260809 01:45:32.784750 10804 leader_election.cc:343] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:32.801921 10596 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_fabd51c93e3347cdb50fb9a28d32ff29" } new_table_name: "default.Table_50f4db524f554499a2dab6d3bc48f053" modify_external_catalogs: true
I20260809 01:45:32.807054 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_8eee5e53a225495ea337c18604165174"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:32.808248 10588 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_8f1bad3be6ed41f086fd8a9cf367555a" } new_table_name: "default.Table_ff0d273341b54d79b4e634ea694e982f" modify_external_catalogs: true
I20260809 01:45:32.810300 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2674c9dbf9e44d63b41427f2fda4e311" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:32.810812 10597 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_421f0008df9b4556b82b41e0c67d277a" } modify_external_catalogs: true
I20260809 01:45:32.811160 11398 raft_consensus.cc:3060] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:32.811579 10597 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_421f0008df9b4556b82b41e0c67d277a" } modify_external_catalogs: true
W20260809 01:45:32.820391 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8eee5e53a225495ea337c18604165174 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:32.828950 10905 raft_consensus.cc:2393] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f60c9ac9414403d9ea5a97b103c2316 in current term 1: Already voted for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in this term.
I20260809 01:45:32.834197 11289 leader_election.cc:304] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:32.837393 11564 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 34) took 1967 ms. Trace:
I20260809 01:45:32.837723 11564 rpcz_store.cc:276] 0809 01:45:30.870322 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:30.870528 (+ 206us) service_pool.cc:224] Handling call
0809 01:45:32.298273 (+1427745us) ts_tablet_manager.cc:1151] Deleting tablet a8c4892ed69143cfb9659a6f4a9d9e55
0809 01:45:32.298328 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:32.370011 (+ 71683us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:32.561703 (+191692us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:32.835944 (+274241us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f19b52 00007f0eeae871d9 00007f0eea09f4e8 00007f0eea167a9e 00007f0eea16aa69 00007f0eea19388e 00007f0eea193859 00007f0eea193815 00007f0eea184a76
0809 01:45:32.837325 (+ 1381us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81370368,"tablet-delete.queue_time_us":1426631}
I20260809 01:45:32.839052 11564 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 36) took 1475 ms. Trace:
I20260809 01:45:32.839391 11564 rpcz_store.cc:276] 0809 01:45:31.363486 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:31.363666 (+ 180us) service_pool.cc:224] Handling call
0809 01:45:32.838731 (+1475065us) ts_tablet_manager.cc:1151] Deleting tablet 43d84f310d8e4df7b8b2d602289025ae
0809 01:45:32.838771 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:32.838999 (+ 228us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1474121}
I20260809 01:45:32.840258 11564 tablet_replica.cc:333] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:32.840950 11553 tablet_bootstrap.cc:654] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:32.840948 11564 raft_consensus.cc:2243] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:32.841713 10586 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_c46005ef8c674ee4819c91c204485828" } modify_external_catalogs: true
I20260809 01:45:32.842478 10586 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_c46005ef8c674ee4819c91c204485828" } modify_external_catalogs: true
I20260809 01:45:32.869668 11760 raft_consensus.cc:493] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:32.870532 11760 raft_consensus.cc:515] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.874873 11760 leader_election.cc:290] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:32.876482 11553 tablet_bootstrap.cc:492] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:32.876895 11553 ts_tablet_manager.cc:1403] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.126s user 0.007s sys 0.007s
I20260809 01:45:32.877120 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ac6aecc93447f7935fbd656fd2a86b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:32.878520 11395 raft_consensus.cc:2468] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 0.
W20260809 01:45:32.880352 10576 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet ead75d766d0346f6823074d90ba8054e consensus state has no leader
I20260809 01:45:32.880389 10805 leader_election.cc:304] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:32.881465 11760 raft_consensus.cc:2804] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:32.882041 11760 raft_consensus.cc:493] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:32.882813 11760 raft_consensus.cc:3060] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:32.888300 11365 tablet_service.cc:1511] Processing CreateTablet for tablet 737d7047a0c74a03b3203112987c0cb2 (DEFAULT_TABLE table=default.Table_25ec70c0a19d4c7dacff1c3a39e55e7e [id=ccd0ea07da604f859047f2eca85fc566]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:32.889458 11365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737d7047a0c74a03b3203112987c0cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:32.893982 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet a8c4892ed69143cfb9659a6f4a9d9e55 (table default.Table_13597d7e311b4ff59dc3782447360563 [id=dbdbdc9a4bd54ba2811644b79f56a451]) successfully deleted
W20260809 01:45:32.895443 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 43d84f310d8e4df7b8b2d602289025ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43d84f310d8e4df7b8b2d602289025ae already in progress: creating tablet
I20260809 01:45:32.898975 11553 raft_consensus.cc:359] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.899725 11553 raft_consensus.cc:385] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:32.905941 11762 consensus_queue.cc:1048] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260809 01:45:32.899961 11553 raft_consensus.cc:740] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:32.914834 11553 consensus_queue.cc:260] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.915650 11760 raft_consensus.cc:515] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.917857 11553 ts_tablet_manager.cc:1434] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.041s user 0.005s sys 0.008s
I20260809 01:45:32.918777 11553 tablet_bootstrap.cc:492] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:32.923923 11398 raft_consensus.cc:2468] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 1.
I20260809 01:45:32.933830 11760 leader_election.cc:290] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:32.932821 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ac6aecc93447f7935fbd656fd2a86b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:32.951261 11392 raft_consensus.cc:3060] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:32.959707 11553 tablet_bootstrap.cc:654] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:32.960078 11655 raft_consensus.cc:2749] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:32.965624 11392 raft_consensus.cc:2468] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 1.
I20260809 01:45:32.967649 10805 leader_election.cc:304] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:32.981451 11655 raft_consensus.cc:493] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:32.982431 11767 raft_consensus.cc:2804] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:32.982177 11655 raft_consensus.cc:515] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.988520 11655 leader_election.cc:290] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:32.984844 10805 leader_election.cc:304] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:32.992508 11760 raft_consensus.cc:2804] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:32.993170 11760 raft_consensus.cc:697] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Becoming Leader. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Running, Role: LEADER
I20260809 01:45:32.993939 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f8824953c794450ba274b160d891fcb" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:32.994669 10905 raft_consensus.cc:2468] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:32.994766 11760 consensus_queue.cc:237] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:32.996598 11289 leader_election.cc:304] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:32.998657 11767 raft_consensus.cc:697] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Becoming Leader. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Running, Role: LEADER
I20260809 01:45:32.999508 11655 raft_consensus.cc:2804] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:33.000080 11655 raft_consensus.cc:493] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:33.000691 11655 raft_consensus.cc:3060] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:33.015579 11723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s user 0.009s sys 0.006s
W20260809 01:45:33.016108 11723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s user 0.009s sys 0.006s
I20260809 01:45:33.025630 11774 raft_consensus.cc:493] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:33.026383 11774 raft_consensus.cc:515] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:33.044301 11551 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.564s user 0.005s sys 0.005s
I20260809 01:45:33.042315 11767 consensus_queue.cc:237] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:33.044618 11551 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.565s user 0.005s sys 0.005s
I20260809 01:45:33.047878 11551 tablet_bootstrap.cc:492] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:33.048398 11551 ts_tablet_manager.cc:1403] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.882s user 0.015s sys 0.008s
I20260809 01:45:33.050282 11774 leader_election.cc:290] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:33.055959 11655 raft_consensus.cc:515] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.062963 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f8824953c794450ba274b160d891fcb" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:33.063657 10905 raft_consensus.cc:3060] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:33.069701 11553 tablet_bootstrap.cc:492] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:33.070479 11554 tablet_bootstrap.cc:654] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:33.071522 11553 ts_tablet_manager.cc:1403] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.153s user 0.009s sys 0.022s
I20260809 01:45:33.075495 11624 leader_election.cc:290] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:33.081657 11553 raft_consensus.cc:359] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.082674 11595 ts_tablet_manager.cc:1929] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:33.083391 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b4850de7d740d794283ca93152fdc1" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:32.841391 11564 raft_consensus.cc:2272] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:33.087332 11564 ts_tablet_manager.cc:1916] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:33.089533 11655 leader_election.cc:290] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:33.094568 11595 log.cc:1199] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/dfb4ad36ca954c7c99407612c0f30711
I20260809 01:45:33.095306 11595 ts_tablet_manager.cc:1950] T dfb4ad36ca954c7c99407612c0f30711 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:33.088099 10905 raft_consensus.cc:2468] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:33.082293 11553 raft_consensus.cc:385] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:33.095964 11553 raft_consensus.cc:740] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:33.096630 11553 consensus_queue.cc:260] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.096972 11289 leader_election.cc:304] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:33.098587 11655 raft_consensus.cc:2804] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:33.098860 11553 ts_tablet_manager.cc:1434] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20260809 01:45:33.099519 11655 raft_consensus.cc:697] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:33.100716 11595 tablet_replica.cc:333] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:33.101444 11595 raft_consensus.cc:2243] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:33.102154 11553 tablet_bootstrap.cc:492] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:33.103890 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet dfb4ad36ca954c7c99407612c0f30711 (table default.Table_a3341f14abab492293b1beadb947e74b [id=7831718cddee4b5b9c42cec19ff7d074]) successfully deleted
I20260809 01:45:33.103554 11655 consensus_queue.cc:237] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.102559 11595 raft_consensus.cc:2272] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:33.112519 11595 ts_tablet_manager.cc:1916] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:33.122613 11553 tablet_bootstrap.cc:654] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:33.134335 11595 ts_tablet_manager.cc:1929] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:33.142269 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b4850de7d740d794283ca93152fdc1" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:33.156450 11595 log.cc:1199] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/61c83b8bcf54461bb24565e3a989100a
I20260809 01:45:33.157191 11595 ts_tablet_manager.cc:1950] T 61c83b8bcf54461bb24565e3a989100a P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:33.161422 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 61c83b8bcf54461bb24565e3a989100a (table default.Table_c46005ef8c674ee4819c91c204485828 [id=8b42fb7f9413434b96d5e788d9919ec2]) successfully deleted
W20260809 01:45:33.162103 11682 consensus_meta.cc:287] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent flushing consensus metadata: real 0.568s user 0.004s sys 0.004s
I20260809 01:45:33.162591 11682 raft_consensus.cc:515] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.166088 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_47a7750fcf1c4cf688f25aff5ded233d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:33.168320 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47a7750fcf1c4cf688f25aff5ded233d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:33.173950 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_e111eb8aa0754bdd9e6e571b06d79add"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:33.176654 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e111eb8aa0754bdd9e6e571b06d79add in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:33.188037 11723 consensus_queue.cc:1048] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:33.189201 11682 leader_election.cc:290] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:33.192929 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af8056e8875c42948923e1d76e49d6d0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
W20260809 01:45:33.194470 10805 leader_election.cc:343] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: Tablet not RUNNING: STOPPED
I20260809 01:45:33.201270 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_98b0f2fc738b4afe8c25c3ff8886c862"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:33.203207 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98b0f2fc738b4afe8c25c3ff8886c862 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:33.213089 11564 ts_tablet_manager.cc:1929] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:33.213441 11564 log.cc:1199] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/af8056e8875c42948923e1d76e49d6d0
I20260809 01:45:33.217778 11564 ts_tablet_manager.cc:1950] T af8056e8875c42948923e1d76e49d6d0 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:33.051939 11551 raft_consensus.cc:359] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.221544 11551 raft_consensus.cc:385] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 01:45:33.144369 11062 consensus_peers.cc:597] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a8c4892ed69143cfb9659a6f4a9d9e55. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:33.201426 11157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ac6aecc93447f7935fbd656fd2a86b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:33.221897 11551 raft_consensus.cc:740] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
W20260809 01:45:33.222093 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 78303.8us
I20260809 01:45:33.058414 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 27) took 2393 ms. Trace:
I20260809 01:45:33.223114 11565 rpcz_store.cc:276] 0809 01:45:30.664717 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:30.664966 (+ 249us) service_pool.cc:224] Handling call
0809 01:45:31.840274 (+1175308us) ts_tablet_manager.cc:1151] Deleting tablet f3c0c5eaa1f64c0298226901de71e7f9
0809 01:45:31.840363 (+ 89us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:32.181338 (+340975us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:32.501867 (+320529us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:33.058331 (+556464us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1174325}
I20260809 01:45:33.223773 11564 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 37) took 1858 ms. Trace:
I20260809 01:45:33.224102 11564 rpcz_store.cc:276] 0809 01:45:31.365269 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:31.365399 (+ 130us) service_pool.cc:224] Handling call
0809 01:45:32.840125 (+1474726us) ts_tablet_manager.cc:1151] Deleting tablet af8056e8875c42948923e1d76e49d6d0
0809 01:45:32.840157 (+ 32us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:33.206137 (+365980us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:33.212983 (+ 6846us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:33.223716 (+ 10733us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1474019}
I20260809 01:45:33.224939 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet f3c0c5eaa1f64c0298226901de71e7f9 (table default.Table_6149470a609f459c951f42acf3332a0d [id=71f0e1b66e0d466ba1297d4ce6c85ad9]) successfully deleted
I20260809 01:45:33.225286 11564 tablet_replica.cc:333] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:33.226054 11565 tablet_replica.cc:333] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:33.226788 11565 raft_consensus.cc:2243] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:33.227722 11565 pending_rounds.cc:70] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7: Trying to abort 1 pending ops.
I20260809 01:45:33.229043 11564 raft_consensus.cc:2243] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:33.229588 11564 pending_rounds.cc:70] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316: Trying to abort 1 pending ops.
I20260809 01:45:33.229782 11564 pending_rounds.cc:77] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438761815449600 op_type: NO_OP noop_request { }
I20260809 01:45:33.231118 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet af8056e8875c42948923e1d76e49d6d0 (table default.Table_90164206f2f44d17895684c1d4dc28da [id=5558f8c7af7a4ad48c8b1ded407d61ba]) successfully deleted
I20260809 01:45:33.231576 11564 raft_consensus.cc:2889] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:33.231921 11564 raft_consensus.cc:2272] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:33.235809 11564 ts_tablet_manager.cc:1916] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:33.238322 10804 leader_election.cc:343] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:33.258388 11564 ts_tablet_manager.cc:1929] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:33.265614 11564 log.cc:1199] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/7e79e27a6c44498b99f363dd654fbb7e
I20260809 01:45:33.266769 11565 pending_rounds.cc:77] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438756342988800 op_type: NO_OP noop_request { }
I20260809 01:45:33.267102 11564 ts_tablet_manager.cc:1950] T 7e79e27a6c44498b99f363dd654fbb7e P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:33.267236 11565 raft_consensus.cc:2889] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:33.267645 11565 raft_consensus.cc:2272] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:33.269502 11565 ts_tablet_manager.cc:1916] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:33.270318 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 7e79e27a6c44498b99f363dd654fbb7e (table default.Table_b9f38ba904ef48559bddb776d4f3a4ff [id=1bdacb31d36c44f58ce3b2264fd8cfcc]) successfully deleted
I20260809 01:45:33.269383 11554 tablet_bootstrap.cc:492] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:33.271916 11554 ts_tablet_manager.cc:1403] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.543s user 0.013s sys 0.004s
W20260809 01:45:33.273932 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 64ac88034c7a4dbb82a39ac7731c37e5 because the tablet was not found. No further retry: Not found: Tablet not found: 64ac88034c7a4dbb82a39ac7731c37e5
I20260809 01:45:33.275092 11554 raft_consensus.cc:359] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.275743 11554 raft_consensus.cc:385] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:33.276007 11554 raft_consensus.cc:740] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:33.227417 11551 consensus_queue.cc:260] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.173709 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ac6aecc93447f7935fbd656fd2a86b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:33.275966 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 34b103e2d3854eb4bf8e5cd649cbe129 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 34b103e2d3854eb4bf8e5cd649cbe129 already in progress: creating tablet
I20260809 01:45:33.188570 11160 raft_consensus.cc:3060] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:33.237111 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af8056e8875c42948923e1d76e49d6d0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:33.277647 11564 tablet_replica.cc:333] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:33.278914 11564 raft_consensus.cc:2243] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:33.238647 11063 leader_election.cc:304] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:33.253976 11157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f8824953c794450ba274b160d891fcb" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:33.280189 11564 raft_consensus.cc:2272] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:33.282120 11157 raft_consensus.cc:2468] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:33.256754 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f8824953c794450ba274b160d891fcb" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:33.284198 11564 ts_tablet_manager.cc:1916] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:33.290380 10596 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 637f0d6261f041b5aeff5fcc709a3dfc has no consensus state
I20260809 01:45:33.293032 10586 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet e5888017345c48c7bafe6f1c6e6945cc
I20260809 01:45:33.293967 11158 raft_consensus.cc:3060] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:33.280778 11554 consensus_queue.cc:260] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.305660 11554 ts_tablet_manager.cc:1434] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.033s user 0.008s sys 0.000s
W20260809 01:45:33.313004 11063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 74440.2us
I20260809 01:45:33.314034 11564 ts_tablet_manager.cc:1929] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:33.316462 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_b6681ecece924ef681c015f56e8cd4ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:33.317312 11564 log.cc:1199] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/dfb4ad36ca954c7c99407612c0f30711
I20260809 01:45:33.318420 11564 ts_tablet_manager.cc:1950] T dfb4ad36ca954c7c99407612c0f30711 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
W20260809 01:45:33.319329 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b6681ecece924ef681c015f56e8cd4ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:33.320058 11554 tablet_bootstrap.cc:492] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:33.322359 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet dfb4ad36ca954c7c99407612c0f30711 (table default.Table_a3341f14abab492293b1beadb947e74b [id=7831718cddee4b5b9c42cec19ff7d074]) successfully deleted
W20260809 01:45:33.325729 11062 consensus_peers.cc:597] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dfb4ad36ca954c7c99407612c0f30711. This is attempt 1: this message will repeat every 5th retry.
W20260809 01:45:33.327445 10804 leader_election.cc:343] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:33.330487 10804 leader_election.cc:343] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:33.331115 10804 leader_election.cc:304] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:33.332988 11063 consensus_peers.cc:597] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dfb4ad36ca954c7c99407612c0f30711. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:33.347584 11682 raft_consensus.cc:2749] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 01:45:33.282363 11551 ts_tablet_manager.cc:1434] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.234s user 0.004s sys 0.003s
I20260809 01:45:33.350054 11551 tablet_bootstrap.cc:492] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:33.359220 11774 raft_consensus.cc:2749] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:33.364665 11158 raft_consensus.cc:2468] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:33.368775 11682 consensus_queue.cc:1048] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:33.371886 11554 tablet_bootstrap.cc:654] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:33.420841 11551 tablet_bootstrap.cc:654] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:33.431638 11554 tablet_bootstrap.cc:492] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:33.432183 11554 ts_tablet_manager.cc:1403] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.126s user 0.015s sys 0.002s
I20260809 01:45:33.449738 11554 raft_consensus.cc:359] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.450604 11554 raft_consensus.cc:385] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:33.450922 11554 raft_consensus.cc:740] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:33.456578 11554 consensus_queue.cc:260] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.491423 10591 master_service.cc:960] ReplaceTablet: received request to replace tablet 5bb5e2ec60c24f8396daed992ca314ac from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:33.496636 10589 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_d0ea8941c02b422a945c246593ba62c3" } new_table_name: "default.Table_57ef64be310d4250b77b141860e944d5" modify_external_catalogs: true
I20260809 01:45:33.501957 10599 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_9590b2028c934cbb91c44edf337bb871" } modify_external_catalogs: true
I20260809 01:45:33.516503 11798 tablet.cc:1724] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:33.526826 10599 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_9590b2028c934cbb91c44edf337bb871" } modify_external_catalogs: true
I20260809 01:45:33.532500 11554 ts_tablet_manager.cc:1434] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.100s user 0.008s sys 0.002s
I20260809 01:45:33.545789 10596 master_service.cc:960] ReplaceTablet: received request to replace tablet b870347b07b04de5b833ab4fd3a2475f from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:33.550109 11551 tablet_bootstrap.cc:492] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:33.550508 11551 ts_tablet_manager.cc:1403] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.201s user 0.034s sys 0.004s
I20260809 01:45:33.560045 10592 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 72b4850de7d740d794283ca93152fdc1 (table default.Table_8f1bad3be6ed41f086fd8a9cf367555a [id=46b096a453084568935bc0d5855cb0f7]). Expected version 2 got 0
I20260809 01:45:33.561604 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 201329c676ec4b6abbb72b22519bc41d (DEFAULT_TABLE table=default.Table_d29473576bc0470bbea9f4ef1fb00e8b [id=b15bb9c641894a39a13c4409f4c51545]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:33.563351 11551 raft_consensus.cc:359] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.564213 11551 raft_consensus.cc:385] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:33.565038 10598 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 5f8824953c794450ba274b160d891fcb (table default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]). Expected version 1 got 0
I20260809 01:45:33.565785 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 201329c676ec4b6abbb72b22519bc41d (DEFAULT_TABLE table=default.Table_d29473576bc0470bbea9f4ef1fb00e8b [id=b15bb9c641894a39a13c4409f4c51545]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:33.565874 11125 tablet_service.cc:1511] Processing CreateTablet for tablet 637f0d6261f041b5aeff5fcc709a3dfc (DEFAULT_TABLE table=default.Table_50f4db524f554499a2dab6d3bc48f053 [id=85ee30d1d77d47f691e6a0dd498d9bca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:33.566692 11551 raft_consensus.cc:740] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:33.566897 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 201329c676ec4b6abbb72b22519bc41d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:33.567395 11551 consensus_queue.cc:260] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.570359 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 201329c676ec4b6abbb72b22519bc41d (DEFAULT_TABLE table=default.Table_d29473576bc0470bbea9f4ef1fb00e8b [id=b15bb9c641894a39a13c4409f4c51545]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:33.571444 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 201329c676ec4b6abbb72b22519bc41d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:33.571995 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 201329c676ec4b6abbb72b22519bc41d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:33.573364 11125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 637f0d6261f041b5aeff5fcc709a3dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:33.573321 10586 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 72b4850de7d740d794283ca93152fdc1 (table default.Table_8f1bad3be6ed41f086fd8a9cf367555a [id=46b096a453084568935bc0d5855cb0f7]). Expected version 2 got 0
I20260809 01:45:33.575249 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 637f0d6261f041b5aeff5fcc709a3dfc (DEFAULT_TABLE table=default.Table_50f4db524f554499a2dab6d3bc48f053 [id=85ee30d1d77d47f691e6a0dd498d9bca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:33.577144 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 637f0d6261f041b5aeff5fcc709a3dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:33.582288 10586 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 5f8824953c794450ba274b160d891fcb (table default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]). Expected version 1 got 0
I20260809 01:45:33.583959 10586 catalog_manager.cc:5719] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65). New cstate: current_term: 1 leader_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:33.586846 10586 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 44ac6aecc93447f7935fbd656fd2a86b (table default.Table_57234dd818414b7397d1ac621bdb06a7 [id=17cbc8bf77e142d08d03bd7a051e00ad]). Expected version 2 got 0
I20260809 01:45:33.590547 10588 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:37956 (request call id 85) took 1108 ms. Trace:
I20260809 01:45:33.595163 11554 tablet_bootstrap.cc:492] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:33.600135 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet a8c4892ed69143cfb9659a6f4a9d9e55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:33.603996 10867 tablet_service.cc:1558] Processing DeleteTablet for tablet af8056e8875c42948923e1d76e49d6d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:33.604944 11595 tablet_replica.cc:333] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:33.605783 11595 raft_consensus.cc:2243] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:33.606326 11595 raft_consensus.cc:2272] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:33.609066 11595 ts_tablet_manager.cc:1916] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:33.608098 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet a8c4892ed69143cfb9659a6f4a9d9e55 because the tablet was not found. No further retry: Not found: Tablet not found: a8c4892ed69143cfb9659a6f4a9d9e55
I20260809 01:45:33.612378 11551 ts_tablet_manager.cc:1434] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.062s user 0.003s sys 0.003s
W20260809 01:45:33.614620 10595 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 21) took 1593 ms (client timeout 1000 ms). Trace:
I20260809 01:45:33.608670 10588 rpcz_store.cc:276] 0809 01:45:32.482276 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:32.482427 (+ 151us) service_pool.cc:224] Handling call
0809 01:45:32.808142 (+325715us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8695846 00007fa2c6252634 00007fa2c62525c2 00007fa2c62524f1 00007fa2c625246c 00007fa2c6251722 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:32.809085 (+ 943us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:32.810060 (+ 975us) catalog_manager.cc:3751] Apply alter schema
0809 01:45:32.811124 (+ 1064us) catalog_manager.cc:3783] Acquired catalog manager lock
0809 01:45:32.811413 (+ 289us) catalog_manager.cc:3941] Updating metadata on disk
0809 01:45:33.014479 (+203066us) catalog_manager.cc:3972] Committing alterations to in-memory state
0809 01:45:33.590456 (+575977us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:32.819855 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:32.820285 (+ 430us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:32.820309 (+ 24us) write_op.cc:435] Acquired schema lock
0809 01:45:32.820323 (+ 14us) tablet.cc:662] Decoding operations
0809 01:45:32.820568 (+ 245us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:32.820608 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:32.820630 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:32.820773 (+ 143us) tablet.cc:701] Row locks acquired
0809 01:45:32.820785 (+ 12us) write_op.cc:260] PREPARE: finished
0809 01:45:32.820902 (+ 117us) write_op.cc:270] Start()
0809 01:45:32.820979 (+ 77us) write_op.cc:276] Timestamp: P: 1786239932820798 usec, L: 0
0809 01:45:32.820995 (+ 16us) op_driver.cc:348] REPLICATION: starting
0809 01:45:32.840624 (+ 19629us) log.cc:844] Serialized 404 byte log entry
0809 01:45:33.010831 (+170207us) op_driver.cc:464] REPLICATION: finished
0809 01:45:33.011382 (+ 551us) write_op.cc:301] APPLY: starting
0809 01:45:33.011474 (+ 92us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:33.011651 (+ 177us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:33.011859 (+ 208us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:33.012074 (+ 215us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:33.012353 (+ 279us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:33.012474 (+ 121us) write_op.cc:312] APPLY: finished
0809 01:45:33.012884 (+ 410us) log.cc:844] Serialized 22 byte log entry
0809 01:45:33.014018 (+ 1134us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:33.014120 (+ 102us) write_op.cc:454] Released schema lock
0809 01:45:33.014182 (+ 62us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:33.015142 (+ 960us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:33.015210 (+ 68us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":746708864,"child_traces":[["op",{"apply.queue_time_us":304,"apply.run_cpu_time_us":2392,"apply.run_wall_time_us":3590,"num_ops":1,"prepare.queue_time_us":6676,"prepare.run_cpu_time_us":2147,"prepare.run_wall_time_us":27215,"raft.queue_time_us":5219,"raft.run_cpu_time_us":18,"raft.run_wall_time_us":18,"replication_time_us":189670}]]}
I20260809 01:45:33.622968 11553 tablet_bootstrap.cc:492] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:33.623714 11553 ts_tablet_manager.cc:1403] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.522s user 0.112s sys 0.187s
I20260809 01:45:33.624209 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 637f0d6261f041b5aeff5fcc709a3dfc (DEFAULT_TABLE table=default.Table_50f4db524f554499a2dab6d3bc48f053 [id=85ee30d1d77d47f691e6a0dd498d9bca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:33.626353 11802 tablet.cc:1724] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:33.628604 11655 raft_consensus.cc:493] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:33.629093 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet e155d16b1d594a789f539390a44bc39f
I20260809 01:45:33.629412 11655 raft_consensus.cc:515] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.630869 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 637f0d6261f041b5aeff5fcc709a3dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:33.633654 11120 tablet_service.cc:1511] Processing CreateTablet for tablet ac6eded297214b58a5d8129a60ca1120 (DEFAULT_TABLE table=default.Table_a0d298080721403d9305bc761dda44a9 [id=96595f8bb6b84744bf7fc74f074a51ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:33.634992 11655 leader_election.cc:290] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:33.636407 11801 tablet.cc:1724] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:33.641459 11124 tablet_service.cc:1558] Processing DeleteTablet for tablet e155d16b1d594a789f539390a44bc39f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:32 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:33.643823 10597 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 88) took 1007 ms. Trace:
W20260809 01:45:33.646425 11289 leader_election.cc:343] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 61c83b8bcf54461bb24565e3a989100a
I20260809 01:45:33.647125 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet e155d16b1d594a789f539390a44bc39f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:32 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:33.647291 11124 tablet_service.cc:1558] Processing DeleteTablet for tablet e155d16b1d594a789f539390a44bc39f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:32 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:33.647799 11564 tablet_replica.cc:333] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:33.647869 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e79e27a6c44498b99f363dd654fbb7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:29 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:33.648516 11564 raft_consensus.cc:2243] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:33.649348 11564 raft_consensus.cc:2272] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:33.649930 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet e155d16b1d594a789f539390a44bc39f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:32 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:33.651156 11564 ts_tablet_manager.cc:1916] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:33.651307 10874 tablet_service.cc:1511] Processing CreateTablet for tablet ac6eded297214b58a5d8129a60ca1120 (DEFAULT_TABLE table=default.Table_a0d298080721403d9305bc761dda44a9 [id=96595f8bb6b84744bf7fc74f074a51ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:33.651382 10588 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_1998ec81597a4c4584dc4e4f90caa274"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:33.652349 10874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac6eded297214b58a5d8129a60ca1120. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:33.652897 10592 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.66:57751 (request call id 20) took 1046 ms (client timeout 1000 ms). Trace:
W20260809 01:45:33.653039 10588 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1998ec81597a4c4584dc4e4f90caa274 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:33.653246 10592 rpcz_store.cc:269] 0809 01:45:32.606899 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:32.607134 (+ 235us) service_pool.cc:224] Handling call
0809 01:45:32.814217 (+207083us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:33.652828 (+838611us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":6,"spinlock_wait_cycles":469636992}
I20260809 01:45:33.653785 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet dfb4ad36ca954c7c99407612c0f30711 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:31 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:33.655884 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_14613c38aa0e4b258598254918b193bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:33.659986 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14613c38aa0e4b258598254918b193bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:33.664790 11365 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e79e27a6c44498b99f363dd654fbb7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:29 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:33.665887 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet a8c4892ed69143cfb9659a6f4a9d9e55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:33.667214 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 7501d985958746bea58e0ff752d7fed8 (DEFAULT_TABLE table=default.Table_03cfb94f08614dc5bcdaf9eb182adc1b [id=d23f238a284a47f2be2faf340b853440]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:33.667841 11369 tablet_service.cc:1511] Processing CreateTablet for tablet ac6eded297214b58a5d8129a60ca1120 (DEFAULT_TABLE table=default.Table_a0d298080721403d9305bc761dda44a9 [id=96595f8bb6b84744bf7fc74f074a51ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:33.670719 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 2674c9dbf9e44d63b41427f2fda4e311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:29 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:33.671844 11365 tablet_service.cc:1558] Processing DeleteTablet for tablet af8056e8875c42948923e1d76e49d6d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:33.673009 11364 tablet_service.cc:1558] Processing DeleteTablet for tablet dfb4ad36ca954c7c99407612c0f30711 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:31 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:33.674283 11129 tablet_service.cc:1511] Processing CreateTablet for tablet da86b5e4a27544c59ae6bb387440d246 (DEFAULT_TABLE table=default.Table_d96a5c9fd7204aaba827a8b16b1c9e62 [id=67ee87fabc7749279341528fbcce16d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:33.674429 10871 tablet_service.cc:1511] Processing CreateTablet for tablet 7501d985958746bea58e0ff752d7fed8 (DEFAULT_TABLE table=default.Table_03cfb94f08614dc5bcdaf9eb182adc1b [id=d23f238a284a47f2be2faf340b853440]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:33.676303 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7501d985958746bea58e0ff752d7fed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:33.676996 11364 tablet_service.cc:1511] Processing CreateTablet for tablet 7501d985958746bea58e0ff752d7fed8 (DEFAULT_TABLE table=default.Table_03cfb94f08614dc5bcdaf9eb182adc1b [id=d23f238a284a47f2be2faf340b853440]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:33.677700 10873 tablet_service.cc:1511] Processing CreateTablet for tablet da86b5e4a27544c59ae6bb387440d246 (DEFAULT_TABLE table=default.Table_d96a5c9fd7204aaba827a8b16b1c9e62 [id=67ee87fabc7749279341528fbcce16d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:33.678682 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da86b5e4a27544c59ae6bb387440d246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:33.678725 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac6eded297214b58a5d8129a60ca1120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:33.681969 11364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7501d985958746bea58e0ff752d7fed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:33.684890 11365 tablet_service.cc:1511] Processing CreateTablet for tablet da86b5e4a27544c59ae6bb387440d246 (DEFAULT_TABLE table=default.Table_d96a5c9fd7204aaba827a8b16b1c9e62 [id=67ee87fabc7749279341528fbcce16d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:33.686322 11365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da86b5e4a27544c59ae6bb387440d246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:33.688982 11553 raft_consensus.cc:359] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:33.660576 10598 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.67:40463 (request call id 13) took 1016 ms (client timeout 1000 ms). Trace:
W20260809 01:45:33.696861 10598 rpcz_store.cc:269] 0809 01:45:32.644692 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:32.958334 (+313642us) service_pool.cc:224] Handling call
0809 01:45:32.971987 (+ 13653us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:33.655810 (+683823us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b4c00002bc0. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c447efc0 00007fa2c448ffa9 00007fa2c449068f 00007fa2c4453e7b 00007fa2c44824dc 00007fa2c448299e 00007fa2c5e1dc1b 00007fa2c85df151 00007fa2c8587e60 00007fa2c858df3b 00007fa2c869453e
0809 01:45:33.660488 (+ 4678us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":8,"spinlock_wait_cycles":26627840}
I20260809 01:45:33.644161 10597 rpcz_store.cc:276] 0809 01:45:32.636579 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:32.637003 (+ 424us) service_pool.cc:224] Handling call
0809 01:45:32.810642 (+173639us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c86954d6 00007fa2c624db34 00007fa2c624dac2 00007fa2c624d9f1 00007fa2c624d96c 00007fa2c624cc22 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:32.812241 (+ 1599us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:32.813332 (+ 1091us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:32.813507 (+ 175us) catalog_manager.cc:2850] Locking tablets
0809 01:45:32.817845 (+ 4338us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:33.051147 (+233302us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:33.051446 (+ 299us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:33.628874 (+577428us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:33.643762 (+ 14888us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:32.851242 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:32.851682 (+ 440us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:32.851700 (+ 18us) write_op.cc:435] Acquired schema lock
0809 01:45:32.851715 (+ 15us) tablet.cc:662] Decoding operations
0809 01:45:32.852015 (+ 300us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:32.852046 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:32.852065 (+ 19us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:32.852316 (+ 251us) tablet.cc:701] Row locks acquired
0809 01:45:32.852329 (+ 13us) write_op.cc:260] PREPARE: finished
0809 01:45:32.852419 (+ 90us) write_op.cc:270] Start()
0809 01:45:32.852482 (+ 63us) write_op.cc:276] Timestamp: P: 1786239932852388 usec, L: 0
0809 01:45:32.852496 (+ 14us) op_driver.cc:348] REPLICATION: starting
0809 01:45:32.853009 (+ 513us) log.cc:844] Serialized 758 byte log entry
0809 01:45:33.033464 (+180455us) op_driver.cc:464] REPLICATION: finished
0809 01:45:33.039295 (+ 5831us) write_op.cc:301] APPLY: starting
0809 01:45:33.039367 (+ 72us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:33.039496 (+ 129us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:33.039596 (+ 100us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:33.039864 (+ 268us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:33.040065 (+ 201us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:33.040130 (+ 65us) write_op.cc:312] APPLY: finished
0809 01:45:33.040350 (+ 220us) log.cc:844] Serialized 28 byte log entry
0809 01:45:33.049990 (+ 9640us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:33.050879 (+ 889us) write_op.cc:454] Released schema lock
0809 01:45:33.050918 (+ 39us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":396413184,"child_traces":[["op",{"apply.queue_time_us":4321,"apply.run_cpu_time_us":1668,"apply.run_wall_time_us":12384,"num_ops":2,"prepare.queue_time_us":12777,"prepare.run_cpu_time_us":2102,"prepare.run_wall_time_us":52166,"raft.queue_time_us":305,"raft.run_cpu_time_us":13,"raft.run_wall_time_us":13,"replication_time_us":180844,"thread_start_us":94889,"threads_started":1}]]}
W20260809 01:45:33.618193 10595 rpcz_store.cc:269] 0809 01:45:32.021464 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:32.021615 (+ 151us) service_pool.cc:224] Handling call
0809 01:45:33.614547 (+1592932us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:32.544508 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:32.545015 (+ 507us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:32.545047 (+ 32us) write_op.cc:435] Acquired schema lock
0809 01:45:32.545065 (+ 18us) tablet.cc:662] Decoding operations
0809 01:45:32.545350 (+ 285us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:32.545399 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:32.545432 (+ 33us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:32.545634 (+ 202us) tablet.cc:701] Row locks acquired
0809 01:45:32.545654 (+ 20us) write_op.cc:260] PREPARE: finished
0809 01:45:32.545783 (+ 129us) write_op.cc:270] Start()
0809 01:45:32.545891 (+ 108us) write_op.cc:276] Timestamp: P: 1786239932545709 usec, L: 0
0809 01:45:32.545912 (+ 21us) op_driver.cc:348] REPLICATION: starting
0809 01:45:32.818151 (+272239us) log.cc:844] Serialized 526 byte log entry
0809 01:45:32.819505 (+ 1354us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678d951 00007fa2c6820208 00007fa2c6caff61 00007fa2c6caf738 00007fa2c6cb43d1 00007fa2c6cb4389 00007fa2c6cb4319 00007fa2c6cb42e1 00007fa2c6cb350d 00000000005bd8c4 00000000005bd6f9
0809 01:45:32.991704 (+172199us) op_driver.cc:464] REPLICATION: finished
0809 01:45:33.005682 (+ 13978us) write_op.cc:301] APPLY: starting
0809 01:45:33.005833 (+ 151us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:33.006018 (+ 185us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:33.006049 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:33.006273 (+ 224us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:33.006419 (+ 146us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:33.006524 (+ 105us) write_op.cc:312] APPLY: finished
0809 01:45:33.006910 (+ 386us) log.cc:844] Serialized 22 byte log entry
0809 01:45:33.007767 (+ 857us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:33.007832 (+ 65us) write_op.cc:454] Released schema lock
0809 01:45:33.007877 (+ 45us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":13644,"apply.run_cpu_time_us":1977,"apply.run_wall_time_us":2547,"num_ops":1,"prepare.queue_time_us":109244,"prepare.run_cpu_time_us":5446,"prepare.run_wall_time_us":275248,"raft.queue_time_us":7764,"raft.run_cpu_time_us":1076,"raft.run_wall_time_us":1075,"replication_time_us":445623,"spinlock_wait_cycles":624684800}]]}
I20260809 01:45:33.695868 11767 raft_consensus.cc:493] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:33.706602 11767 raft_consensus.cc:515] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.710757 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_fe1acd943c5c42c9b09a0a5845cbee68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:33.711536 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d839443474498386220c288ff1721b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:33.712414 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe1acd943c5c42c9b09a0a5845cbee68 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:33.712529 11395 raft_consensus.cc:2468] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 0.
I20260809 01:45:33.714732 11767 leader_election.cc:290] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:33.718261 10599 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet b870347b07b04de5b833ab4fd3a2475f
I20260809 01:45:33.718022 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d839443474498386220c288ff1721b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:33.718133 11157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c83b8bcf54461bb24565e3a989100a" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:33.720499 11283 leader_election.cc:343] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:33.720472 10804 leader_election.cc:343] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:33.721118 11283 leader_election.cc:304] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:33.723018 11655 raft_consensus.cc:2749] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:33.723732 10805 leader_election.cc:304] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:33.724588 11123 tablet_service.cc:1558] Processing DeleteTablet for tablet b870347b07b04de5b833ab4fd3a2475f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:33.726719 11120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac6eded297214b58a5d8129a60ca1120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:33.726891 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet b870347b07b04de5b833ab4fd3a2475f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:33.728086 11551 tablet_bootstrap.cc:492] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:33.730424 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7501d985958746bea58e0ff752d7fed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:33.733635 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da86b5e4a27544c59ae6bb387440d246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:33.741695 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_59150b4860b64269a5f1712f8b8f0d30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:33.742565 10867 tablet_service.cc:1558] Processing DeleteTablet for tablet b870347b07b04de5b833ab4fd3a2475f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:45982
W20260809 01:45:33.743505 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59150b4860b64269a5f1712f8b8f0d30 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:33.743736 11767 raft_consensus.cc:2804] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:33.744302 11767 raft_consensus.cc:493] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:33.744845 11767 raft_consensus.cc:3060] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:33.746304 10591 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 5bb5e2ec60c24f8396daed992ca314ac
I20260809 01:45:33.748867 10591 catalog_manager.cc:6807] ReplaceTablet: tablet 5bb5e2ec60c24f8396daed992ca314ac deleted and replaced by tablet c2f86bba4bd94fbcacdc9c139e9de2d7
I20260809 01:45:33.751174 11123 tablet_service.cc:1558] Processing DeleteTablet for tablet 5bb5e2ec60c24f8396daed992ca314ac with delete_type TABLET_DATA_DELETED (replaced by tablet c2f86bba4bd94fbcacdc9c139e9de2d7) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:33.758028 10867 tablet_service.cc:1558] Processing DeleteTablet for tablet 5bb5e2ec60c24f8396daed992ca314ac with delete_type TABLET_DATA_DELETED (replaced by tablet c2f86bba4bd94fbcacdc9c139e9de2d7) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:33.774489 11553 raft_consensus.cc:385] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:33.775698 11553 raft_consensus.cc:740] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:33.776990 11553 consensus_queue.cc:260] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.779589 11565 ts_tablet_manager.cc:1929] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:33.780002 11565 log.cc:1199] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/825c568f98014fbf92ade7bd99763581
I20260809 01:45:33.780925 11565 ts_tablet_manager.cc:1950] T 825c568f98014fbf92ade7bd99763581 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
W20260809 01:45:33.790576 10589 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 12fb852b8ea8478cb160a265459865c4 consensus state has no leader
I20260809 01:45:33.796290 11553 ts_tablet_manager.cc:1434] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.172s user 0.007s sys 0.000s
I20260809 01:45:33.805552 11553 tablet_bootstrap.cc:492] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:33.826102 10589 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_3239fc2ae3d348c39baafc0778a13a41" } new_table_name: "default.Table_fb9b15123c274d7db877eac3e7906edc" modify_external_catalogs: true
I20260809 01:45:33.847756 11551 tablet_bootstrap.cc:654] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:33.848984 11767 raft_consensus.cc:515] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:33.861678 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 5bb5e2ec60c24f8396daed992ca314ac with delete_type TABLET_DATA_DELETED (replaced by tablet c2f86bba4bd94fbcacdc9c139e9de2d7) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:33.865610 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet 2674c9dbf9e44d63b41427f2fda4e311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:29 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:33.868685 11767 leader_election.cc:290] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:33.872735 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 28) took 3146 ms. Trace:
I20260809 01:45:33.873081 11565 rpcz_store.cc:276] 0809 01:45:30.726013 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:30.727497 (+ 1484us) service_pool.cc:224] Handling call
0809 01:45:33.225864 (+2498367us) ts_tablet_manager.cc:1151] Deleting tablet 825c568f98014fbf92ade7bd99763581
0809 01:45:33.225946 (+ 82us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:33.414052 (+188106us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:33.779471 (+365419us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:33.872672 (+ 93201us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2497481}
I20260809 01:45:33.874375 11565 tablet_replica.cc:333] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:33.875037 11565 raft_consensus.cc:2243] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:33.875486 11565 raft_consensus.cc:2272] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:33.885953 11553 tablet_bootstrap.cc:654] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:33.886133 10576 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 12fb852b8ea8478cb160a265459865c4 (table default.Table_57ef64be310d4250b77b141860e944d5 [id=212edfae1bf849c3b46674ec191ee76d], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 12fb852b8ea8478cb160a265459865c4 consensus state has no leader
I20260809 01:45:33.889843 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 825c568f98014fbf92ade7bd99763581 (table default.Table_006a6ebb9836470396c69b4b78abc954 [id=366d143256e84961b8b8ae2bc08452d0]) successfully deleted
I20260809 01:45:33.902212 10586 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_4eef5dd402bc41168cd7b02c32e84835" } modify_external_catalogs: true
I20260809 01:45:33.903143 10586 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_4eef5dd402bc41168cd7b02c32e84835" } modify_external_catalogs: true
I20260809 01:45:33.908564 11553 tablet_bootstrap.cc:492] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:33.909055 11553 ts_tablet_manager.cc:1403] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.104s user 0.009s sys 0.017s
I20260809 01:45:33.920866 11553 raft_consensus.cc:359] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:33.921728 11553 raft_consensus.cc:385] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:33.922113 11553 raft_consensus.cc:740] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:33.923005 11553 consensus_queue.cc:260] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:33.926371 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d839443474498386220c288ff1721b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
W20260809 01:45:33.928444 10804 leader_election.cc:343] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:33.930989 10592 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_e111eb8aa0754bdd9e6e571b06d79add" } modify_external_catalogs: true
I20260809 01:45:33.931512 10592 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_e111eb8aa0754bdd9e6e571b06d79add" } modify_external_catalogs: true
I20260809 01:45:33.933558 11553 ts_tablet_manager.cc:1434] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.024s user 0.007s sys 0.004s
I20260809 01:45:33.925422 11565 ts_tablet_manager.cc:1916] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:33.935179 11553 tablet_bootstrap.cc:492] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:33.942610 11564 ts_tablet_manager.cc:1929] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:33.943058 11564 log.cc:1199] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/e155d16b1d594a789f539390a44bc39f
I20260809 01:45:33.944031 11564 ts_tablet_manager.cc:1950] T e155d16b1d594a789f539390a44bc39f P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:33.945835 11595 ts_tablet_manager.cc:1929] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:33.946182 11595 log.cc:1199] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/af8056e8875c42948923e1d76e49d6d0
I20260809 01:45:33.946902 11595 ts_tablet_manager.cc:1950] T af8056e8875c42948923e1d76e49d6d0 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:33.956099 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d839443474498386220c288ff1721b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:33.959544 11595 tablet_replica.cc:333] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:33.963622 11767 consensus_queue.cc:1048] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:33.965463 11767 consensus_queue.cc:1048] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:33.966272 11553 tablet_bootstrap.cc:654] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:33.956864 11398 raft_consensus.cc:3060] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:33.969892 11595 raft_consensus.cc:2243] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:33.975449 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet e155d16b1d594a789f539390a44bc39f (table default.Table_421f0008df9b4556b82b41e0c67d277a [id=966459ae200d45429a01326bc653f8d3]) successfully deleted
I20260809 01:45:33.985244 11595 pending_rounds.cc:70] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8: Trying to abort 1 pending ops.
W20260809 01:45:33.990042 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 7e79e27a6c44498b99f363dd654fbb7e because the tablet was not found. No further retry: Not found: Tablet not found: 7e79e27a6c44498b99f363dd654fbb7e
I20260809 01:45:33.987432 11595 pending_rounds.cc:77] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438761815449600 op_type: NO_OP noop_request { }
I20260809 01:45:33.993498 11553 tablet_bootstrap.cc:492] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:33.994269 11553 ts_tablet_manager.cc:1403] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.059s user 0.017s sys 0.011s
I20260809 01:45:33.997294 10595 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_25ec70c0a19d4c7dacff1c3a39e55e7e" } new_table_name: "default.Table_f499e027721646b7a747c9b576e672d5" modify_external_catalogs: true
I20260809 01:45:33.998210 11553 raft_consensus.cc:359] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:33.993973 11595 raft_consensus.cc:2889] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:33.998842 11553 raft_consensus.cc:385] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:33.999163 11553 raft_consensus.cc:740] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:33.999773 11553 consensus_queue.cc:260] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:34.003546 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet a8c4892ed69143cfb9659a6f4a9d9e55 because the tablet was not found. No further retry: Not found: Tablet not found: a8c4892ed69143cfb9659a6f4a9d9e55
I20260809 01:45:34.003661 11595 raft_consensus.cc:2272] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:34.008440 11564 tablet_replica.cc:333] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:34.009753 11553 ts_tablet_manager.cc:1434] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260809 01:45:33.949203 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet af8056e8875c42948923e1d76e49d6d0 (table default.Table_90164206f2f44d17895684c1d4dc28da [id=5558f8c7af7a4ad48c8b1ded407d61ba]) successfully deleted
I20260809 01:45:34.017971 11553 tablet_bootstrap.cc:492] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
W20260809 01:45:34.017863 10580 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 68694us
I20260809 01:45:34.022059 10598 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 72b4850de7d740d794283ca93152fdc1 (table default.Table_ff0d273341b54d79b4e634ea694e982f [id=46b096a453084568935bc0d5855cb0f7]). Expected version 3 got 2
I20260809 01:45:34.024061 11551 tablet_bootstrap.cc:492] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:34.024466 11551 ts_tablet_manager.cc:1403] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.297s user 0.006s sys 0.016s
I20260809 01:45:34.026890 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_dc1ac1b0a9db479fb1ec9416455c008d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:34.028821 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc1ac1b0a9db479fb1ec9416455c008d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:34.013820 11398 raft_consensus.cc:2468] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 1.
I20260809 01:45:34.032863 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_e53beaa5c2fe49e3ba6eaa8181b5106a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:34.034974 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e53beaa5c2fe49e3ba6eaa8181b5106a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:34.038018 11551 raft_consensus.cc:359] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:34.038807 11551 raft_consensus.cc:385] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:34.039103 11551 raft_consensus.cc:740] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:34.039862 11551 consensus_queue.cc:260] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:34.042062 10805 leader_election.cc:304] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:34.043804 11624 raft_consensus.cc:2804] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:34.044423 11624 raft_consensus.cc:697] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Becoming Leader. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Running, Role: LEADER
W20260809 01:45:34.046054 10804 consensus_peers.cc:597] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:34.045778 11624 consensus_queue.cc:237] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:34.047569 10874 tablet_service.cc:1558] Processing DeleteTablet for tablet 43d84f310d8e4df7b8b2d602289025ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:34.049862 11595 ts_tablet_manager.cc:1916] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:34.052523 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_8031d30fd4be45869a2e535707f3459b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:34.053637 11564 raft_consensus.cc:2243] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:34.055626 11564 pending_rounds.cc:70] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316: Trying to abort 1 pending ops.
I20260809 01:45:34.065670 11564 pending_rounds.cc:77] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438765562736640 op_type: NO_OP noop_request { }
I20260809 01:45:34.066242 11564 raft_consensus.cc:2889] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
W20260809 01:45:34.067207 10804 consensus_peers.cc:597] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:34.068073 11564 raft_consensus.cc:2272] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:34.073988 11682 raft_consensus.cc:493] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:34.074640 11682 raft_consensus.cc:515] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:34.078379 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8031d30fd4be45869a2e535707f3459b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:34.087667 11565 ts_tablet_manager.cc:1929] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:34.088155 11565 log.cc:1199] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/af8056e8875c42948923e1d76e49d6d0
I20260809 01:45:34.088795 11565 ts_tablet_manager.cc:1950] T af8056e8875c42948923e1d76e49d6d0 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:34.095839 11551 ts_tablet_manager.cc:1434] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.071s user 0.006s sys 0.001s
I20260809 01:45:34.096716 11551 tablet_bootstrap.cc:492] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:34.098162 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 32) took 2726 ms. Trace:
I20260809 01:45:34.098433 11565 rpcz_store.cc:276] 0809 01:45:31.371404 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:31.371553 (+ 149us) service_pool.cc:224] Handling call
0809 01:45:33.874191 (+2502638us) ts_tablet_manager.cc:1151] Deleting tablet af8056e8875c42948923e1d76e49d6d0
0809 01:45:33.874268 (+ 77us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:33.956603 (+ 82335us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:34.087546 (+130943us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:34.098109 (+ 10563us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2501787}
I20260809 01:45:34.099913 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 31) took 2729 ms. Trace:
I20260809 01:45:34.100220 11565 rpcz_store.cc:276] 0809 01:45:31.370896 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:31.373630 (+ 2734us) service_pool.cc:224] Handling call
0809 01:45:34.099452 (+2725822us) ts_tablet_manager.cc:1151] Deleting tablet 43d84f310d8e4df7b8b2d602289025ae
0809 01:45:34.099515 (+ 63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:34.099865 (+ 350us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2724809}
I20260809 01:45:34.101158 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet af8056e8875c42948923e1d76e49d6d0 (table default.Table_90164206f2f44d17895684c1d4dc28da [id=5558f8c7af7a4ad48c8b1ded407d61ba]) successfully deleted
W20260809 01:45:34.102418 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 43d84f310d8e4df7b8b2d602289025ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43d84f310d8e4df7b8b2d602289025ae already in progress: creating tablet
I20260809 01:45:34.101404 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 40) took 1649 ms. Trace:
I20260809 01:45:34.120237 11565 rpcz_store.cc:276] 0809 01:45:32.451669 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:32.452997 (+ 1328us) service_pool.cc:224] Handling call
0809 01:45:34.101098 (+1648101us) ts_tablet_manager.cc:1151] Deleting tablet 34b103e2d3854eb4bf8e5cd649cbe129
0809 01:45:34.101132 (+ 34us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:34.101351 (+ 219us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1646891}
I20260809 01:45:34.121394 11565 tablet_replica.cc:333] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
W20260809 01:45:34.121897 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 34b103e2d3854eb4bf8e5cd649cbe129 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 34b103e2d3854eb4bf8e5cd649cbe129 already in progress: creating tablet
I20260809 01:45:34.122095 11565 raft_consensus.cc:2243] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:34.122696 11564 ts_tablet_manager.cc:1916] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:34.122953 11565 raft_consensus.cc:2272] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:34.131103 11551 tablet_bootstrap.cc:654] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:34.133404 11554 tablet_bootstrap.cc:654] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:34.124809 11565 ts_tablet_manager.cc:1916] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:34.147588 11682 leader_election.cc:290] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:34.148289 11595 ts_tablet_manager.cc:1929] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:34.152269 11595 log.cc:1199] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/7e79e27a6c44498b99f363dd654fbb7e
I20260809 01:45:34.154062 11595 ts_tablet_manager.cc:1950] T 7e79e27a6c44498b99f363dd654fbb7e P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
W20260809 01:45:34.143522 10805 consensus_peers.cc:597] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:34.167255 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d84f310d8e4df7b8b2d602289025ae" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:34.169482 10804 leader_election.cc:343] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:34.169855 10870 consensus_meta.cc:287] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent flushing consensus metadata: real 0.549s user 0.007s sys 0.007s
I20260809 01:45:34.171797 11553 tablet_bootstrap.cc:654] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:34.172770 11595 tablet_replica.cc:333] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:34.174243 11595 raft_consensus.cc:2243] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:34.174830 11595 raft_consensus.cc:2272] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:34.176777 11595 ts_tablet_manager.cc:1916] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:34.181499 11655 consensus_queue.cc:1048] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:34.191848 11551 tablet_bootstrap.cc:492] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:34.192333 11551 ts_tablet_manager.cc:1403] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.096s user 0.012s sys 0.008s
I20260809 01:45:34.200367 11551 raft_consensus.cc:359] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:34.201103 11551 raft_consensus.cc:385] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:34.201417 11551 raft_consensus.cc:740] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:34.202217 11551 consensus_queue.cc:260] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:34.217430 11551 ts_tablet_manager.cc:1434] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.025s user 0.005s sys 0.005s
I20260809 01:45:34.222406 11551 tablet_bootstrap.cc:492] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:34.239919 11551 tablet_bootstrap.cc:654] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:34.241783 11802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:45982 (request call id 51) took 1136 ms. Trace:
I20260809 01:45:34.252990 11655 consensus_queue.cc:1048] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:34.264082 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 7e79e27a6c44498b99f363dd654fbb7e (table default.Table_b9f38ba904ef48559bddb776d4f3a4ff [id=1bdacb31d36c44f58ce3b2264fd8cfcc]) successfully deleted
I20260809 01:45:34.262442 11802 rpcz_store.cc:276] 0809 01:45:33.105259 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:33.105498 (+ 239us) service_pool.cc:224] Handling call
0809 01:45:34.241685 (+1136187us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:33.124568 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0809 01:45:33.127462 (+ 2894us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 01:45:33.127521 (+ 59us) alter_schema_op.cc:66] Acquired schema lock
0809 01:45:33.127617 (+ 96us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0809 01:45:33.127763 (+ 146us) alter_schema_op.cc:128] START. Timestamp: 1786239933126770
0809 01:45:33.127822 (+ 59us) op_driver.cc:348] REPLICATION: starting
0809 01:45:33.130502 (+ 2680us) log.cc:844] Serialized 173 byte log entry
0809 01:45:33.499988 (+369486us) op_driver.cc:464] REPLICATION: finished
0809 01:45:33.607423 (+107435us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0809 01:45:33.698432 (+ 91009us) log.cc:844] Serialized 5 byte log entry
0809 01:45:34.219248 (+520816us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:34.239687 (+ 20439us) log.cc:844] Serialized 14 byte log entry
0809 01:45:34.239918 (+ 231us) alter_schema_op.cc:73] Released schema lock
0809 01:45:34.241555 (+ 1637us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":44385,"prepare.queue_time_us":16888,"prepare.run_cpu_time_us":5649,"prepare.run_wall_time_us":368677,"raft.queue_time_us":254944,"raft.run_cpu_time_us":8411,"raft.run_wall_time_us":54484,"replication_time_us":371908,"thread_start_us":387491,"threads_started":3,"wal-append.queue_time_us":113249}]]}
I20260809 01:45:34.269760 11551 tablet_bootstrap.cc:492] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:34.270165 11551 ts_tablet_manager.cc:1403] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.048s user 0.008s sys 0.004s
I20260809 01:45:34.272873 11551 raft_consensus.cc:359] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:34.273571 11551 raft_consensus.cc:385] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:34.273890 11551 raft_consensus.cc:740] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:34.274600 11551 consensus_queue.cc:260] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:34.276386 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d84f310d8e4df7b8b2d602289025ae" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:34.283200 11354 consensus_meta.cc:287] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.640s user 0.006s sys 0.005s
I20260809 01:45:34.293048 11551 ts_tablet_manager.cc:1434] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
W20260809 01:45:34.294550 10805 leader_election.cc:343] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:34.295076 10805 leader_election.cc:304] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:34.299278 11551 tablet_bootstrap.cc:492] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:34.305553 11624 raft_consensus.cc:2749] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:34.319346 11553 tablet_bootstrap.cc:492] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:34.319818 11553 ts_tablet_manager.cc:1403] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.302s user 0.014s sys 0.011s
I20260809 01:45:34.330920 11553 raft_consensus.cc:359] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:34.331638 11553 raft_consensus.cc:385] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:34.331908 11553 raft_consensus.cc:740] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:34.335254 11565 ts_tablet_manager.cc:1929] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:34.341822 11565 log.cc:1199] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/dfb4ad36ca954c7c99407612c0f30711
I20260809 01:45:34.342558 11565 ts_tablet_manager.cc:1950] T dfb4ad36ca954c7c99407612c0f30711 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:34.344389 11553 consensus_queue.cc:260] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:34.346297 11553 ts_tablet_manager.cc:1434] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.026s user 0.004s sys 0.003s
I20260809 01:45:34.350090 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 41) took 1897 ms. Trace:
I20260809 01:45:34.350397 11565 rpcz_store.cc:276] 0809 01:45:32.452682 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:32.454421 (+ 1739us) service_pool.cc:224] Handling call
0809 01:45:34.121262 (+1666841us) ts_tablet_manager.cc:1151] Deleting tablet dfb4ad36ca954c7c99407612c0f30711
0809 01:45:34.121299 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:34.260336 (+139037us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:34.335125 (+ 74789us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:34.350032 (+ 14907us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1665803}
I20260809 01:45:34.351016 11551 tablet_bootstrap.cc:654] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:34.350301 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 55766.9us
W20260809 01:45:34.352499 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 102764us
I20260809 01:45:34.355312 11553 tablet_bootstrap.cc:492] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:34.365216 11802 tablet.cc:1724] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 01:45:34.370769 11595 ts_tablet_manager.cc:1929] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:34.371158 11595 log.cc:1199] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/e155d16b1d594a789f539390a44bc39f
I20260809 01:45:34.371845 11595 ts_tablet_manager.cc:1950] T e155d16b1d594a789f539390a44bc39f P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:34.382624 11842 raft_consensus.cc:493] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 629f002048f64491afdd57f72a466e59)
I20260809 01:45:34.383244 11842 raft_consensus.cc:515] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:34.410274 11554 tablet_bootstrap.cc:492] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:34.411454 11801 tablet.cc:1724] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260809 01:45:34.422998 10581 consensus_peers.cc:597] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 -> Peer 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063): Couldn't send request to peer 71bc72deb1d2402cabd817dfe3b27e88. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260809 01:45:34.428217 11564 ts_tablet_manager.cc:1929] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:34.454934 11564 log.cc:1199] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/2674c9dbf9e44d63b41427f2fda4e311
I20260809 01:45:34.433581 11554 ts_tablet_manager.cc:1403] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.839s user 0.015s sys 0.005s
I20260809 01:45:34.459473 11554 raft_consensus.cc:359] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:34.462222 11564 ts_tablet_manager.cc:1950] T 2674c9dbf9e44d63b41427f2fda4e311 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:34.463492 11595 tablet_replica.cc:333] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:34.464025 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet e155d16b1d594a789f539390a44bc39f (table default.Table_421f0008df9b4556b82b41e0c67d277a [id=966459ae200d45429a01326bc653f8d3]) successfully deleted
W20260809 01:45:34.465677 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet dfb4ad36ca954c7c99407612c0f30711 because the tablet was not found. No further retry: Not found: Tablet not found: dfb4ad36ca954c7c99407612c0f30711
W20260809 01:45:34.466794 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet b870347b07b04de5b833ab4fd3a2475f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b870347b07b04de5b833ab4fd3a2475f already in progress: creating tablet
W20260809 01:45:34.475553 10589 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d0091542071441e5bad0c2dca819f37f consensus state has no leader
I20260809 01:45:34.478328 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet dfb4ad36ca954c7c99407612c0f30711 (table default.Table_a3341f14abab492293b1beadb947e74b [id=7831718cddee4b5b9c42cec19ff7d074]) successfully deleted
W20260809 01:45:34.479807 10595 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 737d7047a0c74a03b3203112987c0cb2 consensus state has no leader
I20260809 01:45:34.481360 10595 master_service.cc:960] ReplaceTablet: received request to replace tablet 12fb852b8ea8478cb160a265459865c4 from {username='slave'} at 127.0.0.1:37956
W20260809 01:45:34.482134 10320 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111) [suppressed 13 similar messages]
I20260809 01:45:34.479775 10589 master_service.cc:960] ReplaceTablet: received request to replace tablet da86b5e4a27544c59ae6bb387440d246 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:34.477732 11565 tablet_replica.cc:333] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:34.490257 11595 raft_consensus.cc:2243] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:34.490784 11595 raft_consensus.cc:2272] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:34.494279 11823 raft_consensus.cc:493] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:34.460345 11554 raft_consensus.cc:385] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:34.500947 11554 raft_consensus.cc:740] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:34.501855 11554 consensus_queue.cc:260] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:34.503952 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 2674c9dbf9e44d63b41427f2fda4e311 (table default.Table_7f7d5bf058bd48ff8e3660ca567a5718 [id=0806119aff0647e7aee1fc802147e2f5]) successfully deleted
W20260809 01:45:34.504788 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet af8056e8875c42948923e1d76e49d6d0 because the tablet was not found. No further retry: Not found: Tablet not found: af8056e8875c42948923e1d76e49d6d0
W20260809 01:45:34.505456 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet dfb4ad36ca954c7c99407612c0f30711 because the tablet was not found. No further retry: Not found: Tablet not found: dfb4ad36ca954c7c99407612c0f30711
W20260809 01:45:34.506139 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet b870347b07b04de5b833ab4fd3a2475f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b870347b07b04de5b833ab4fd3a2475f already in progress: creating tablet
W20260809 01:45:34.506830 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 5bb5e2ec60c24f8396daed992ca314ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5bb5e2ec60c24f8396daed992ca314ac already in progress: creating tablet
W20260809 01:45:34.513780 10320 leader_election.cc:336] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063): Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
I20260809 01:45:34.497193 11823 raft_consensus.cc:515] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:34.521970 11595 ts_tablet_manager.cc:1916] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:34.525667 11554 ts_tablet_manager.cc:1434] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.070s user 0.009s sys 0.000s
I20260809 01:45:34.525717 11565 raft_consensus.cc:2243] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:34.527288 11554 tablet_bootstrap.cc:492] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:34.527729 11551 tablet_bootstrap.cc:492] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:34.528215 11551 ts_tablet_manager.cc:1403] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.229s user 0.011s sys 0.006s
I20260809 01:45:34.532047 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09a73ee8e8ec496a984efcab27715506" candidate_term: 3 candidate_status { last_received { term: 2 index: 127 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:45:34.536275 11802 tablet.cc:1719] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8: Skipping requested alter to schema version 3, tablet already version 3
I20260809 01:45:34.537259 11565 raft_consensus.cc:2272] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:34.538439 10319 leader_election.cc:304] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09a73ee8e8ec496a984efcab27715506; no voters: 629f002048f64491afdd57f72a466e59, 71bc72deb1d2402cabd817dfe3b27e88
I20260809 01:45:34.544762 11565 ts_tablet_manager.cc:1916] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:34.524700 11858 tablet.cc:1724] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 01:45:34.546950 11551 raft_consensus.cc:359] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:34.554000 11842 leader_election.cc:290] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:34.565879 11862 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 01:45:34.578260 11551 raft_consensus.cc:385] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:34.596242 10599 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233]). Expected version 2 got 0
I20260809 01:45:34.598313 10599 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 98bee541fc1345fa83ffebeba05d8c8e (table default.Table_e212a7cfad494f2d97ac5eada6ac15f5 [id=d2c591183b4646ca88b53698200a62c2]). Expected version 1 got 0
I20260809 01:45:34.619297 11823 leader_election.cc:290] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:34.628540 11553 tablet_bootstrap.cc:654] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:34.613436 11565 ts_tablet_manager.cc:1929] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:34.624110 11120 tablet_service.cc:1511] Processing CreateTablet for tablet 8f0d79e03e66410487a72f1b47b327a0 (DEFAULT_TABLE table=default.Table_b6681ecece924ef681c015f56e8cd4ae [id=869fc1e71c7142c5bbd1fa358c94e453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:34.630659 11565 log.cc:1199] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/a8c4892ed69143cfb9659a6f4a9d9e55
I20260809 01:45:34.610152 10599 catalog_manager.cc:5719] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65). New cstate: current_term: 1 leader_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:34.631376 11120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0d79e03e66410487a72f1b47b327a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:34.631656 11565 ts_tablet_manager.cc:1950] T a8c4892ed69143cfb9659a6f4a9d9e55 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:34.633656 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 42) took 2177 ms. Trace:
I20260809 01:45:34.634085 11565 rpcz_store.cc:276] 0809 01:45:32.455947 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:32.456116 (+ 169us) service_pool.cc:224] Handling call
0809 01:45:34.477376 (+2021260us) ts_tablet_manager.cc:1151] Deleting tablet a8c4892ed69143cfb9659a6f4a9d9e55
0809 01:45:34.477496 (+ 120us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:34.559775 (+ 82279us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:34.613313 (+ 53538us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:34.633597 (+ 20284us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2020527}
I20260809 01:45:34.635553 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 43) took 2085 ms. Trace:
I20260809 01:45:34.635869 11565 rpcz_store.cc:276] 0809 01:45:32.550118 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:32.550264 (+ 146us) service_pool.cc:224] Handling call
0809 01:45:34.635191 (+2084927us) ts_tablet_manager.cc:1151] Deleting tablet 61c83b8bcf54461bb24565e3a989100a
0809 01:45:34.635229 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:34.635504 (+ 275us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1967402}
I20260809 01:45:34.636909 11565 tablet_replica.cc:333] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:34.644294 11801 tablet.cc:1719] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316: Skipping requested alter to schema version 3, tablet already version 3
I20260809 01:45:34.630400 11551 raft_consensus.cc:740] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:34.646945 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ead75d766d0346f6823074d90ba8054e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:34.651022 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 8f0d79e03e66410487a72f1b47b327a0 (DEFAULT_TABLE table=default.Table_b6681ecece924ef681c015f56e8cd4ae [id=869fc1e71c7142c5bbd1fa358c94e453]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:34.655927 11848 log.cc:927] Time spent T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8: Append to log took a long time: real 0.158s user 0.001s sys 0.000s
I20260809 01:45:34.653276 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0d79e03e66410487a72f1b47b327a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:34.656574 11802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:45982 (request call id 57) took 1009 ms. Trace:
W20260809 01:45:34.658814 10805 leader_election.cc:343] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:34.657063 11802 rpcz_store.cc:276] 0809 01:45:33.647158 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:33.647544 (+ 386us) service_pool.cc:224] Handling call
0809 01:45:33.699568 (+ 52024us) spinlock_profiling.cc:243] Waited 50 ms on lock 0x7b5c0008ff0c. stack: 00007fe5ab55ecf3 00007fe5ab55ebf0 00007fe5ab10c574 0000000000516de4 0000000000516d79 00007fe5b2ab5b41 00007fe5b2c5b874 00007fe5b374db1d 00007fe5b1ce16f4 00007fe5b1ce1682 00007fe5b1ce15b1 00007fe5b1ce152c 00007fe5b1ce07e2 00007fe5b03c107c 00007fe5b03c04b6 00007fe5b03bfe5f
0809 01:45:34.656511 (+956943us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:34.241110 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0809 01:45:34.241460 (+ 350us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 01:45:34.455262 (+213802us) alter_schema_op.cc:66] Acquired schema lock
0809 01:45:34.455360 (+ 98us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0809 01:45:34.455504 (+ 144us) alter_schema_op.cc:128] START. Timestamp: 1786239934455601
0809 01:45:34.455552 (+ 48us) op_driver.cc:348] REPLICATION: starting
0809 01:45:34.456473 (+ 921us) log.cc:844] Serialized 173 byte log entry
0809 01:45:34.535823 (+ 79350us) op_driver.cc:464] REPLICATION: finished
0809 01:45:34.536131 (+ 308us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0809 01:45:34.582302 (+ 46171us) log.cc:844] Serialized 94 byte log entry
0809 01:45:34.582499 (+ 197us) alter_schema_op.cc:73] Released schema lock
0809 01:45:34.656380 (+ 73881us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"spinlock_wait_cycles":115047296,"child_traces":[["op",{"apply.queue_time_us":158,"prepare.queue_time_us":540929,"prepare.run_cpu_time_us":10696,"prepare.run_wall_time_us":215704,"replication_time_us":80160}]]}
I20260809 01:45:34.635444 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet a8c4892ed69143cfb9659a6f4a9d9e55 (table default.Table_13597d7e311b4ff59dc3782447360563 [id=dbdbdc9a4bd54ba2811644b79f56a451]) successfully deleted
I20260809 01:45:34.664026 11801 tablet.cc:1724] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:34.668190 10871 tablet_service.cc:1511] Processing CreateTablet for tablet 8f0d79e03e66410487a72f1b47b327a0 (DEFAULT_TABLE table=default.Table_b6681ecece924ef681c015f56e8cd4ae [id=869fc1e71c7142c5bbd1fa358c94e453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:34.669422 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0d79e03e66410487a72f1b47b327a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:34.678134 11802 tablet.cc:1724] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:34.637616 11565 raft_consensus.cc:2243] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:34.678833 10596 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 5f8824953c794450ba274b160d891fcb (table default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]). Expected version 1 got 0
I20260809 01:45:34.687299 10596 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 72b4850de7d740d794283ca93152fdc1 (table default.Table_ff0d273341b54d79b4e634ea694e982f [id=46b096a453084568935bc0d5855cb0f7]). Expected version 3 got 2
I20260809 01:45:34.694032 11554 tablet_bootstrap.cc:654] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:34.657923 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ead75d766d0346f6823074d90ba8054e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:34.647688 11551 consensus_queue.cc:260] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:34.700589 11565 raft_consensus.cc:2272] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:34.702317 11595 ts_tablet_manager.cc:1929] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:34.704798 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 59de95452c0a4945beae5a9344537e0f (DEFAULT_TABLE table=default.Table_8eee5e53a225495ea337c18604165174 [id=a178ba9920604ce988ec5c4fca1bec93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:34.711452 11551 ts_tablet_manager.cc:1434] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.183s user 0.011s sys 0.000s
I20260809 01:45:34.712844 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59de95452c0a4945beae5a9344537e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:34.724560 11565 ts_tablet_manager.cc:1916] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:34.736709 11551 tablet_bootstrap.cc:492] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:34.754987 11815 raft_consensus.cc:493] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:34.755787 11815 raft_consensus.cc:515] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:34.768208 11815 leader_election.cc:290] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:34.784292 11551 tablet_bootstrap.cc:654] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:34.796028 11120 tablet_service.cc:1511] Processing CreateTablet for tablet 135d3d65ea7846d3bc2cb59d44dbedf6 (DEFAULT_TABLE table=default.Table_f2379f27ebbb4266ab36dafd7d80f79f [id=18e87d9e1fa3442aaa489e3abcd4640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:34.796404 11815 raft_consensus.cc:493] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:34.797830 11120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 135d3d65ea7846d3bc2cb59d44dbedf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:34.797355 11815 raft_consensus.cc:515] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:34.803395 11815 leader_election.cc:290] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:34.809998 11858 tablet.cc:1719] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7: Skipping requested alter to schema version 3, tablet already version 3
I20260809 01:45:34.822335 11565 ts_tablet_manager.cc:1929] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:34.822693 11565 log.cc:1199] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/e155d16b1d594a789f539390a44bc39f
I20260809 01:45:34.706168 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 59de95452c0a4945beae5a9344537e0f (DEFAULT_TABLE table=default.Table_8eee5e53a225495ea337c18604165174 [id=a178ba9920604ce988ec5c4fca1bec93]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:34.712458 10804 leader_election.cc:343] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:34.823550 11565 ts_tablet_manager.cc:1950] T e155d16b1d594a789f539390a44bc39f P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:34.823768 11595 log.cc:1199] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/5bb5e2ec60c24f8396daed992ca314ac
I20260809 01:45:34.824465 11595 ts_tablet_manager.cc:1950] T 5bb5e2ec60c24f8396daed992ca314ac P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:34.823856 10804 leader_election.cc:304] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:34.824486 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59de95452c0a4945beae5a9344537e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:34.825080 11624 raft_consensus.cc:2749] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:34.825376 10804 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 112993us
I20260809 01:45:34.826290 11595 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 65) took 1073 ms. Trace:
I20260809 01:45:34.748826 11553 tablet_bootstrap.cc:492] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:34.775665 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2674c9dbf9e44d63b41427f2fda4e311" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:34.781085 11837 raft_consensus.cc:493] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:34.801952 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d839443474498386220c288ff1721b" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:34.828346 11553 ts_tablet_manager.cc:1403] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.473s user 0.018s sys 0.013s
I20260809 01:45:34.828372 11837 raft_consensus.cc:515] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:34.831815 11837 leader_election.cc:290] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:34.834350 11553 raft_consensus.cc:359] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:34.835110 11553 raft_consensus.cc:385] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:34.835374 11553 raft_consensus.cc:740] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:34.836112 11553 consensus_queue.cc:260] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:34.846154 11157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bee541fc1345fa83ffebeba05d8c8e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:34.849052 11553 ts_tablet_manager.cc:1434] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.020s user 0.006s sys 0.002s
W20260809 01:45:34.849984 10804 leader_election.cc:343] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:34.844864 11815 raft_consensus.cc:493] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:34.855255 10871 tablet_service.cc:1511] Processing CreateTablet for tablet 135d3d65ea7846d3bc2cb59d44dbedf6 (DEFAULT_TABLE table=default.Table_f2379f27ebbb4266ab36dafd7d80f79f [id=18e87d9e1fa3442aaa489e3abcd4640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:34.858484 11553 tablet_bootstrap.cc:492] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:34.853514 11125 tablet_service.cc:1511] Processing CreateTablet for tablet 389407e7ff464e0d8e84bbdbc4c6081e (DEFAULT_TABLE table=default.Table_47a7750fcf1c4cf688f25aff5ded233d [id=1fb02cbd4f2d4807b326a8fff08316d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:34.870222 11125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389407e7ff464e0d8e84bbdbc4c6081e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:34.826573 11595 rpcz_store.cc:276] 0809 01:45:33.753244 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:33.757652 (+ 4408us) service_pool.cc:224] Handling call
0809 01:45:34.463350 (+705698us) ts_tablet_manager.cc:1151] Deleting tablet 5bb5e2ec60c24f8396daed992ca314ac
0809 01:45:34.463386 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:34.610610 (+147224us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:34.702164 (+ 91554us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:34.826228 (+124064us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":704610}
I20260809 01:45:34.872202 11595 tablet_replica.cc:333] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:34.872907 11595 raft_consensus.cc:2243] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:34.873853 11595 raft_consensus.cc:2272] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:34.862092 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 135d3d65ea7846d3bc2cb59d44dbedf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:34.887753 11595 ts_tablet_manager.cc:1916] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:34.888906 10598 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_d96a5c9fd7204aaba827a8b16b1c9e62" } new_table_name: "default.Table_29a4bd6bd10a473dbd9dbda008b9eb16" modify_external_catalogs: true
I20260809 01:45:34.893857 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_5264c640dfc04d9bae010805cdaf4ea6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:34.895437 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5264c640dfc04d9bae010805cdaf4ea6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:34.895593 11553 tablet_bootstrap.cc:654] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:34.898725 10592 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet a9d1ed215ba644abbae09e78016b2c21
I20260809 01:45:34.899464 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 5bb5e2ec60c24f8396daed992ca314ac (table default.Table_5f4d617a86c2487fa9f64cb58f14de3c [id=f64daa5a2706436081e9166a5f3a0cd7]) successfully deleted
I20260809 01:45:34.902525 10592 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_dc1ac1b0a9db479fb1ec9416455c008d" } new_table_name: "default.Table_6d4e0b3afc864450bdf1a8fb35f7dc75" modify_external_catalogs: true
I20260809 01:45:34.908857 10586 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet c795f81d2bc248eeb39b64d6aaa0370c
I20260809 01:45:34.910482 10586 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 103) took 1009 ms. Trace:
I20260809 01:45:34.916091 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet da86b5e4a27544c59ae6bb387440d246
I20260809 01:45:34.917594 10589 catalog_manager.cc:6807] ReplaceTablet: tablet da86b5e4a27544c59ae6bb387440d246 deleted and replaced by tablet 60d356fcc48441ca80f2df29b533404f
I20260809 01:45:34.921178 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet c795f81d2bc248eeb39b64d6aaa0370c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:34.920814 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet a9d1ed215ba644abbae09e78016b2c21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:34.910836 10586 rpcz_store.cc:276] 0809 01:45:33.901544 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:33.901713 (+ 169us) service_pool.cc:224] Handling call
0809 01:45:33.904119 (+ 2406us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:33.905863 (+ 1744us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:33.906113 (+ 250us) catalog_manager.cc:2850] Locking tablets
0809 01:45:34.602750 (+696637us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:34.908363 (+305613us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:34.908558 (+ 195us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:34.908665 (+ 107us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:34.910429 (+ 1764us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:34.874763 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:34.875154 (+ 391us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:34.875178 (+ 24us) write_op.cc:435] Acquired schema lock
0809 01:45:34.875190 (+ 12us) tablet.cc:662] Decoding operations
0809 01:45:34.875484 (+ 294us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:34.875519 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:34.875539 (+ 20us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:34.875804 (+ 265us) tablet.cc:701] Row locks acquired
0809 01:45:34.875816 (+ 12us) write_op.cc:260] PREPARE: finished
0809 01:45:34.875923 (+ 107us) write_op.cc:270] Start()
0809 01:45:34.876017 (+ 94us) write_op.cc:276] Timestamp: P: 1786239934875867 usec, L: 0
0809 01:45:34.876049 (+ 32us) op_driver.cc:348] REPLICATION: starting
0809 01:45:34.876547 (+ 498us) log.cc:844] Serialized 725 byte log entry
0809 01:45:34.889161 (+ 12614us) op_driver.cc:464] REPLICATION: finished
0809 01:45:34.904131 (+ 14970us) write_op.cc:301] APPLY: starting
0809 01:45:34.904195 (+ 64us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:34.904297 (+ 102us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:34.904374 (+ 77us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:34.904575 (+ 201us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:34.904699 (+ 124us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:34.904736 (+ 37us) write_op.cc:312] APPLY: finished
0809 01:45:34.904912 (+ 176us) log.cc:844] Serialized 29 byte log entry
0809 01:45:34.907865 (+ 2953us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:34.907926 (+ 61us) write_op.cc:454] Released schema lock
0809 01:45:34.907959 (+ 33us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":14803,"mutex_wait_us":10687,"num_ops":2,"prepare.queue_time_us":252107,"prepare.run_cpu_time_us":2016,"prepare.run_wall_time_us":5817,"replication_time_us":13023}]]}
I20260809 01:45:34.923209 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet da86b5e4a27544c59ae6bb387440d246 with delete_type TABLET_DATA_DELETED (replaced by tablet 60d356fcc48441ca80f2df29b533404f) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:34.924420 11129 tablet_service.cc:1558] Processing DeleteTablet for tablet a9d1ed215ba644abbae09e78016b2c21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:34.925148 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet c795f81d2bc248eeb39b64d6aaa0370c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:34.925902 11123 tablet_service.cc:1558] Processing DeleteTablet for tablet da86b5e4a27544c59ae6bb387440d246 with delete_type TABLET_DATA_DELETED (replaced by tablet 60d356fcc48441ca80f2df29b533404f) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:34.926108 11551 tablet_bootstrap.cc:492] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:34.926635 11551 ts_tablet_manager.cc:1403] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.194s user 0.011s sys 0.003s
I20260809 01:45:34.926709 11126 tablet_service.cc:1558] Processing DeleteTablet for tablet 825c568f98014fbf92ade7bd99763581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:34.929134 11551 raft_consensus.cc:359] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:34.937803 11120 tablet_service.cc:1511] Processing CreateTablet for tablet ef8a334741d3419eaa377338e0eaa889 (DEFAULT_TABLE table=default.Table_f6654879d18843fd84d44ea5f170dda7 [id=51771c6c664942e4b1df1132b5641ffa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:34.938328 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 389407e7ff464e0d8e84bbdbc4c6081e (DEFAULT_TABLE table=default.Table_47a7750fcf1c4cf688f25aff5ded233d [id=1fb02cbd4f2d4807b326a8fff08316d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:34.938952 11120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef8a334741d3419eaa377338e0eaa889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:34.939571 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389407e7ff464e0d8e84bbdbc4c6081e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:34.941900 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 50) took 1300 ms. Trace:
I20260809 01:45:34.942348 11565 rpcz_store.cc:276] 0809 01:45:33.641093 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:33.641272 (+ 179us) service_pool.cc:224] Handling call
0809 01:45:34.636750 (+995478us) ts_tablet_manager.cc:1151] Deleting tablet e155d16b1d594a789f539390a44bc39f
0809 01:45:34.636788 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:34.723817 (+ 87029us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b20000477b0. stack: 00007fe4bc3e2cf3 00007fe4bc3e2bf0 00007fe4bbf90574 0000000000516de4 0000000000516d79 00007fe4c453e211 00007fe4c456ed43 00007fe4c45767b2 00007fe4c4576731 00007fe4c45766c1 00007fe4c4576681 00007fe4c45753ed 00000000005bd8c4 00000000005bd6f9 00007fe4bc368faf 00007fe4c3915446
0809 01:45:34.753815 (+ 29998us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:34.822194 (+ 68379us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:34.941825 (+119631us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23311744,"tablet-delete.queue_time_us":994743}
I20260809 01:45:34.944075 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet e155d16b1d594a789f539390a44bc39f (table default.Table_421f0008df9b4556b82b41e0c67d277a [id=966459ae200d45429a01326bc653f8d3]) successfully deleted
I20260809 01:45:34.945393 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 51) took 1298 ms. Trace:
I20260809 01:45:34.946406 11553 tablet_bootstrap.cc:492] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:34.946826 11553 ts_tablet_manager.cc:1403] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.089s user 0.008s sys 0.004s
I20260809 01:45:34.951759 10595 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 12fb852b8ea8478cb160a265459865c4
I20260809 01:45:34.952461 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_54419467eb324360916beed8ea56f0db"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:34.953487 10595 catalog_manager.cc:6807] ReplaceTablet: tablet 12fb852b8ea8478cb160a265459865c4 deleted and replaced by tablet 7706838afc9f4dbebc3213b82c4da126
I20260809 01:45:34.953974 11553 raft_consensus.cc:359] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:34.954249 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_54419467eb324360916beed8ea56f0db in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:34.954586 11553 raft_consensus.cc:385] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:34.954816 11553 raft_consensus.cc:740] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:34.955410 11553 consensus_queue.cc:260] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:34.958535 11595 ts_tablet_manager.cc:1929] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:34.958932 11595 log.cc:1199] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/2674c9dbf9e44d63b41427f2fda4e311
I20260809 01:45:34.959607 11595 ts_tablet_manager.cc:1950] T 2674c9dbf9e44d63b41427f2fda4e311 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
W20260809 01:45:34.963389 10596 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.66:57751 (request call id 21) took 1237 ms (client timeout 1000 ms). Trace:
W20260809 01:45:34.964362 10596 rpcz_store.cc:269] 0809 01:45:33.725999 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:33.731187 (+ 5188us) service_pool.cc:224] Handling call
0809 01:45:34.963295 (+1232108us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":8}
I20260809 01:45:34.958478 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet 12fb852b8ea8478cb160a265459865c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 7706838afc9f4dbebc3213b82c4da126) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:34.971432 11553 ts_tablet_manager.cc:1434] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.024s user 0.003s sys 0.004s
I20260809 01:45:34.972152 11353 tablet_service.cc:1511] Processing CreateTablet for tablet ef8a334741d3419eaa377338e0eaa889 (DEFAULT_TABLE table=default.Table_f6654879d18843fd84d44ea5f170dda7 [id=51771c6c664942e4b1df1132b5641ffa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:34.972520 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 59de95452c0a4945beae5a9344537e0f (DEFAULT_TABLE table=default.Table_8eee5e53a225495ea337c18604165174 [id=a178ba9920604ce988ec5c4fca1bec93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:34.982213 10871 tablet_service.cc:1511] Processing CreateTablet for tablet ef8a334741d3419eaa377338e0eaa889 (DEFAULT_TABLE table=default.Table_f6654879d18843fd84d44ea5f170dda7 [id=51771c6c664942e4b1df1132b5641ffa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:34.981777 11352 tablet_service.cc:1558] Processing DeleteTablet for tablet 12fb852b8ea8478cb160a265459865c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 7706838afc9f4dbebc3213b82c4da126) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:34.982862 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet c795f81d2bc248eeb39b64d6aaa0370c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:34.983451 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef8a334741d3419eaa377338e0eaa889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:34.920277 11815 raft_consensus.cc:515] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:34.956734 11126 tablet_service.cc:1558] Processing DeleteTablet for tablet 12fb852b8ea8478cb160a265459865c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 7706838afc9f4dbebc3213b82c4da126) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:34.959790 11123 tablet_service.cc:1558] Processing DeleteTablet for tablet dfb4ad36ca954c7c99407612c0f30711 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:31 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:34.986132 11565 rpcz_store.cc:276] 0809 01:45:33.646986 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:33.647139 (+ 153us) service_pool.cc:224] Handling call
0809 01:45:34.945013 (+1297874us) ts_tablet_manager.cc:1151] Deleting tablet e155d16b1d594a789f539390a44bc39f
0809 01:45:34.945058 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:34.945339 (+ 281us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1297349}
I20260809 01:45:34.987130 11815 leader_election.cc:290] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:34.969302 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet a8c4892ed69143cfb9659a6f4a9d9e55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:28 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:34.969798 11129 tablet_service.cc:1558] Processing DeleteTablet for tablet af8056e8875c42948923e1d76e49d6d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:34.989763 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ac6aecc93447f7935fbd656fd2a86b" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:34.929847 11551 raft_consensus.cc:385] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:34.991938 11551 raft_consensus.cc:740] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:34.993701 11360 tablet_service.cc:1558] Processing DeleteTablet for tablet da86b5e4a27544c59ae6bb387440d246 with delete_type TABLET_DATA_DELETED (replaced by tablet 60d356fcc48441ca80f2df29b533404f) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:34.995071 11553 tablet_bootstrap.cc:492] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:35.000072 11553 tablet_bootstrap.cc:654] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:34.968468 10576 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 12fb852b8ea8478cb160a265459865c4 (table default.Table_57ef64be310d4250b77b141860e944d5 [id=212edfae1bf849c3b46674ec191ee76d], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 12fb852b8ea8478cb160a265459865c4 consensus state has no leader
I20260809 01:45:35.002290 11365 tablet_service.cc:1511] Processing CreateTablet for tablet 389407e7ff464e0d8e84bbdbc4c6081e (DEFAULT_TABLE table=default.Table_47a7750fcf1c4cf688f25aff5ded233d [id=1fb02cbd4f2d4807b326a8fff08316d8]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:35.002583 10576 catalog_manager.cc:5017] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet e155d16b1d594a789f539390a44bc39f because the tablet was not found. No further retry: Not found: Tablet not found: e155d16b1d594a789f539390a44bc39f
I20260809 01:45:35.003960 11129 tablet_service.cc:1511] Processing CreateTablet for tablet 7df6343423604a00a46f203f61e98e2f (DEFAULT_TABLE table=default.Table_98b0f2fc738b4afe8c25c3ff8886c862 [id=b7fa92e91f85479c82ba9d384c44d429]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:35.005239 11595 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 66) took 1140 ms. Trace:
I20260809 01:45:35.013906 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 7df6343423604a00a46f203f61e98e2f (DEFAULT_TABLE table=default.Table_98b0f2fc738b4afe8c25c3ff8886c862 [id=b7fa92e91f85479c82ba9d384c44d429]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:35.015021 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7df6343423604a00a46f203f61e98e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:35.015839 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 4754f2e2a35f4a6a91fa333e2cf564e4 (DEFAULT_TABLE table=default.Table_d1d088df5e7d452ab5a8771a8d3679dd [id=bd9dd208af1e4bd596ae7e3a4b219faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:35.016196 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d839443474498386220c288ff1721b" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:35.017603 11392 raft_consensus.cc:2393] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33b5652538e64ed58a09993d3188f8d7 in current term 1: Already voted for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in this term.
I20260809 01:45:35.018544 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet a9d1ed215ba644abbae09e78016b2c21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:35.019022 11062 leader_election.cc:304] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:35.020067 11815 raft_consensus.cc:3060] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:35.022680 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_080ef38220da4f5f8f016c1f733ee2ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:35.024366 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_080ef38220da4f5f8f016c1f733ee2ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:35.026013 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_837bb268533c42268f543bcc9732f18e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:35.026541 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bee541fc1345fa83ffebeba05d8c8e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:35.027724 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_837bb268533c42268f543bcc9732f18e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:35.027993 10805 leader_election.cc:343] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:35.028421 10805 leader_election.cc:304] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:35.032523 11837 raft_consensus.cc:2749] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:35.034771 11123 tablet_service.cc:1511] Processing CreateTablet for tablet a9d1ed215ba644abbae09e78016b2c21 (DEFAULT_TABLE table=default.Table_e111eb8aa0754bdd9e6e571b06d79add [id=78fdf345e656424e9612369430887930]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:35.032454 11815 raft_consensus.cc:2749] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:35.035140 11364 tablet_service.cc:1511] Processing CreateTablet for tablet 135d3d65ea7846d3bc2cb59d44dbedf6 (DEFAULT_TABLE table=default.Table_f2379f27ebbb4266ab36dafd7d80f79f [id=18e87d9e1fa3442aaa489e3abcd4640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:35.036772 11837 consensus_queue.cc:1048] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:35.041822 11595 rpcz_store.cc:276] 0809 01:45:33.865168 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:33.865401 (+ 233us) service_pool.cc:224] Handling call
0809 01:45:34.872056 (+1006655us) ts_tablet_manager.cc:1151] Deleting tablet 2674c9dbf9e44d63b41427f2fda4e311
0809 01:45:34.872098 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:34.894595 (+ 22497us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:34.958420 (+ 63825us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:35.005171 (+ 46751us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1005864}
I20260809 01:45:35.042986 11595 tablet_replica.cc:333] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:35.042940 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ac6aecc93447f7935fbd656fd2a86b" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:35.043705 11595 raft_consensus.cc:2243] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:35.044123 11595 raft_consensus.cc:2272] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:35.046456 11062 leader_election.cc:304] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:35.054376 10598 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 60d356fcc48441ca80f2df29b533404f has no consensus state
I20260809 01:45:35.057389 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef8a334741d3419eaa377338e0eaa889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:35.061147 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 2674c9dbf9e44d63b41427f2fda4e311 (table default.Table_7f7d5bf058bd48ff8e3660ca567a5718 [id=0806119aff0647e7aee1fc802147e2f5]) successfully deleted
W20260809 01:45:35.062337 10592 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c16897857df24c6da5744a99d5f687c0 has no consensus state
I20260809 01:45:35.064078 11595 ts_tablet_manager.cc:1916] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:35.065562 11369 tablet_service.cc:1511] Processing CreateTablet for tablet 7df6343423604a00a46f203f61e98e2f (DEFAULT_TABLE table=default.Table_98b0f2fc738b4afe8c25c3ff8886c862 [id=b7fa92e91f85479c82ba9d384c44d429]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:35.067709 10874 tablet_service.cc:1511] Processing CreateTablet for tablet a9d1ed215ba644abbae09e78016b2c21 (DEFAULT_TABLE table=default.Table_e111eb8aa0754bdd9e6e571b06d79add [id=78fdf345e656424e9612369430887930]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:35.068851 10874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9d1ed215ba644abbae09e78016b2c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:35.071208 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 4754f2e2a35f4a6a91fa333e2cf564e4 (DEFAULT_TABLE table=default.Table_d1d088df5e7d452ab5a8771a8d3679dd [id=bd9dd208af1e4bd596ae7e3a4b219faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:35.081561 11364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 135d3d65ea7846d3bc2cb59d44dbedf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:35.085248 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59de95452c0a4945beae5a9344537e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:35.088523 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4754f2e2a35f4a6a91fa333e2cf564e4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:35.094499 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 12fb852b8ea8478cb160a265459865c4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12fb852b8ea8478cb160a265459865c4 already in progress: creating tablet
W20260809 01:45:35.095340 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet c795f81d2bc248eeb39b64d6aaa0370c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c795f81d2bc248eeb39b64d6aaa0370c already in progress: creating tablet
W20260809 01:45:35.095954 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet da86b5e4a27544c59ae6bb387440d246 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da86b5e4a27544c59ae6bb387440d246 already in progress: creating tablet
W20260809 01:45:35.096554 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet a9d1ed215ba644abbae09e78016b2c21 because the tablet was not found. No further retry: Not found: Tablet not found: a9d1ed215ba644abbae09e78016b2c21
I20260809 01:45:35.096894 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 4754f2e2a35f4a6a91fa333e2cf564e4 (DEFAULT_TABLE table=default.Table_d1d088df5e7d452ab5a8771a8d3679dd [id=bd9dd208af1e4bd596ae7e3a4b219faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:35.098068 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4754f2e2a35f4a6a91fa333e2cf564e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:35.098714 11553 tablet_bootstrap.cc:492] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:35.099076 11553 ts_tablet_manager.cc:1403] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.104s user 0.000s sys 0.008s
I20260809 01:45:35.101325 11553 raft_consensus.cc:359] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.101951 11553 raft_consensus.cc:385] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:35.102136 11365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389407e7ff464e0d8e84bbdbc4c6081e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:35.102216 11553 raft_consensus.cc:740] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:35.102864 11553 consensus_queue.cc:260] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.104694 11553 ts_tablet_manager.cc:1434] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 01:45:35.109478 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7df6343423604a00a46f203f61e98e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:35.110296 10589 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_d37c43f0673044eb826f3944101bcd4b" } modify_external_catalogs: true
I20260809 01:45:35.110890 10589 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_d37c43f0673044eb826f3944101bcd4b" } modify_external_catalogs: true
W20260809 01:45:35.113121 11062 leader_election.cc:343] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Not found: Tablet not found: 2674c9dbf9e44d63b41427f2fda4e311
I20260809 01:45:35.113587 11062 leader_election.cc:304] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:35.117276 11366 tablet_service.cc:1511] Processing CreateTablet for tablet a9d1ed215ba644abbae09e78016b2c21 (DEFAULT_TABLE table=default.Table_e111eb8aa0754bdd9e6e571b06d79add [id=78fdf345e656424e9612369430887930]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:35.118546 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9d1ed215ba644abbae09e78016b2c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:35.126822 10595 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_e212a7cfad494f2d97ac5eada6ac15f5" } modify_external_catalogs: true
I20260809 01:45:35.127396 10595 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_e212a7cfad494f2d97ac5eada6ac15f5" } modify_external_catalogs: true
I20260809 01:45:35.141390 11553 tablet_bootstrap.cc:492] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:35.148782 10592 master_service.cc:960] ReplaceTablet: received request to replace tablet 98bee541fc1345fa83ffebeba05d8c8e from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:35.152055 11595 ts_tablet_manager.cc:1929] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:35.152415 11595 log.cc:1199] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/43d84f310d8e4df7b8b2d602289025ae
I20260809 01:45:35.153057 11595 ts_tablet_manager.cc:1950] T 43d84f310d8e4df7b8b2d602289025ae P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:35.154906 11595 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 67) took 1108 ms. Trace:
I20260809 01:45:35.155205 11595 rpcz_store.cc:276] 0809 01:45:34.047049 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:34.047206 (+ 157us) service_pool.cc:224] Handling call
0809 01:45:35.042836 (+995630us) ts_tablet_manager.cc:1151] Deleting tablet 43d84f310d8e4df7b8b2d602289025ae
0809 01:45:35.042880 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:35.091830 (+ 48950us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:35.151934 (+ 60104us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:35.154847 (+ 2913us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":994710}
I20260809 01:45:35.156462 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 43d84f310d8e4df7b8b2d602289025ae (table default.Table_2712961a5fbc40689aeb5e6dcbf6d79f [id=2a521d855351481299fb19d2b1f99b37]) successfully deleted
I20260809 01:45:35.146842 11553 tablet_bootstrap.cc:654] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:35.160447 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet c795f81d2bc248eeb39b64d6aaa0370c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c795f81d2bc248eeb39b64d6aaa0370c already in progress: creating tablet
W20260809 01:45:35.162262 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet a9d1ed215ba644abbae09e78016b2c21 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9d1ed215ba644abbae09e78016b2c21 already in progress: creating tablet
W20260809 01:45:35.163918 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet da86b5e4a27544c59ae6bb387440d246 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da86b5e4a27544c59ae6bb387440d246 already in progress: creating tablet
W20260809 01:45:35.165575 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 12fb852b8ea8478cb160a265459865c4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12fb852b8ea8478cb160a265459865c4 already in progress: creating tablet
I20260809 01:45:35.171725 11553 tablet_bootstrap.cc:492] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:35.172158 11553 ts_tablet_manager.cc:1403] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.031s user 0.008s sys 0.004s
I20260809 01:45:35.174671 11553 raft_consensus.cc:359] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.175400 11553 raft_consensus.cc:385] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:35.175679 11553 raft_consensus.cc:740] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:35.176301 11553 consensus_queue.cc:260] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.178206 11553 ts_tablet_manager.cc:1434] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 01:45:35.194226 11815 raft_consensus.cc:2749] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:35.194803 11815 raft_consensus.cc:2749] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:35.197275 11553 tablet_bootstrap.cc:492] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:35.203313 11553 tablet_bootstrap.cc:654] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:35.207931 11837 raft_consensus.cc:493] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:35.208441 11837 raft_consensus.cc:515] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.209213 10591 master_service.cc:960] ReplaceTablet: received request to replace tablet 12fb852b8ea8478cb160a265459865c4 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:35.214046 11391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d0a9d73d4841c393e7d2cef41fee89" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:35.215443 11837 leader_election.cc:290] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:35.217626 10805 leader_election.cc:343] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:35.228539 11553 tablet_bootstrap.cc:492] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:35.229040 11553 ts_tablet_manager.cc:1403] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.032s user 0.013s sys 0.002s
I20260809 01:45:35.232846 11553 raft_consensus.cc:359] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.233559 11553 raft_consensus.cc:385] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:35.233996 11553 raft_consensus.cc:740] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:35.234925 11553 consensus_queue.cc:260] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.232646 10588 catalog_manager.cc:5719] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67). New cstate: current_term: 1 leader_uuid: "4f60c9ac9414403d9ea5a97b103c2316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:35.238480 10588 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 5f8824953c794450ba274b160d891fcb (table default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]). Expected version 1 got 0
I20260809 01:45:35.240747 10588 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 72b4850de7d740d794283ca93152fdc1 (table default.Table_ff0d273341b54d79b4e634ea694e982f [id=46b096a453084568935bc0d5855cb0f7]). Expected version 3 got 2
I20260809 01:45:35.246212 10588 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 44ac6aecc93447f7935fbd656fd2a86b (table default.Table_57234dd818414b7397d1ac621bdb06a7 [id=17cbc8bf77e142d08d03bd7a051e00ad]). Expected version 2 got 0
I20260809 01:45:35.250025 11553 ts_tablet_manager.cc:1434] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.021s user 0.004s sys 0.003s
I20260809 01:45:35.252092 11160 raft_consensus.cc:3060] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:35.252969 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 54) took 1529 ms. Trace:
I20260809 01:45:35.253392 11565 rpcz_store.cc:276] 0809 01:45:33.724152 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:33.724291 (+ 139us) service_pool.cc:224] Handling call
0809 01:45:34.987877 (+1263586us) ts_tablet_manager.cc:1151] Deleting tablet b870347b07b04de5b833ab4fd3a2475f
0809 01:45:34.987919 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:35.252899 (+264980us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1262734}
W20260809 01:45:35.255172 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet b870347b07b04de5b833ab4fd3a2475f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b870347b07b04de5b833ab4fd3a2475f already in progress: creating tablet
I20260809 01:45:34.992993 11551 consensus_queue.cc:260] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:35.257934 11551 ts_tablet_manager.cc:1434] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.331s user 0.008s sys 0.000s
I20260809 01:45:35.257727 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d0a9d73d4841c393e7d2cef41fee89" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:35.258313 11815 raft_consensus.cc:493] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:35.259022 11815 raft_consensus.cc:515] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:35.260887 10804 leader_election.cc:343] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:35.261413 10804 leader_election.cc:304] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:35.263170 11837 consensus_queue.cc:1048] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:35.264959 11837 raft_consensus.cc:2749] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:35.269191 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7df6343423604a00a46f203f61e98e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:35.275995 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_e39314f303ee429f897d4f85b8edbf69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:35.278697 11553 tablet_bootstrap.cc:492] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
W20260809 01:45:35.286438 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e39314f303ee429f897d4f85b8edbf69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:35.294412 11553 tablet_bootstrap.cc:654] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:35.294687 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4754f2e2a35f4a6a91fa333e2cf564e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:35.300104 10592 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_f499e027721646b7a747c9b576e672d5" } new_table_name: "default.Table_cf7bd2b9aec04c73b8ed72debcab795d" modify_external_catalogs: true
I20260809 01:45:35.313230 11553 tablet_bootstrap.cc:492] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:35.313868 11553 ts_tablet_manager.cc:1403] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.035s user 0.012s sys 0.010s
I20260809 01:45:35.315709 11123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9d1ed215ba644abbae09e78016b2c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:35.321986 11553 raft_consensus.cc:359] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.322639 11553 raft_consensus.cc:385] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:35.322897 11553 raft_consensus.cc:740] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
W20260809 01:45:35.311920 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 198852us
I20260809 01:45:35.323475 11553 consensus_queue.cc:260] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.329793 11565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 55) took 1579 ms. Trace:
I20260809 01:45:35.330168 11565 rpcz_store.cc:276] 0809 01:45:33.750795 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:33.750964 (+ 169us) service_pool.cc:224] Handling call
0809 01:45:35.256007 (+1505043us) ts_tablet_manager.cc:1151] Deleting tablet 5bb5e2ec60c24f8396daed992ca314ac
0809 01:45:35.326205 (+ 70198us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:35.329739 (+ 3534us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1504287}
W20260809 01:45:35.332613 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 5bb5e2ec60c24f8396daed992ca314ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5bb5e2ec60c24f8396daed992ca314ac already in progress: creating tablet
I20260809 01:45:35.347362 11391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d84f310d8e4df7b8b2d602289025ae" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:35.347934 11391 raft_consensus.cc:2468] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:35.349254 11062 leader_election.cc:304] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:35.350101 11888 raft_consensus.cc:2804] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:35.350454 11888 raft_consensus.cc:493] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:35.350755 11888 raft_consensus.cc:3060] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:35.365482 11554 tablet_bootstrap.cc:492] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:35.366341 11554 ts_tablet_manager.cc:1403] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.839s user 0.010s sys 0.014s
W20260809 01:45:35.370450 10576 catalog_manager.cc:5017] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet a9d1ed215ba644abbae09e78016b2c21 because the tablet was not found. No further retry: Not found: Tablet not found: a9d1ed215ba644abbae09e78016b2c21
W20260809 01:45:35.371367 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet c795f81d2bc248eeb39b64d6aaa0370c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c795f81d2bc248eeb39b64d6aaa0370c already in progress: creating tablet
I20260809 01:45:35.373726 11624 raft_consensus.cc:493] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:35.374423 11624 raft_consensus.cc:515] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.378378 11624 leader_election.cc:290] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:35.380828 11553 ts_tablet_manager.cc:1434] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.067s user 0.003s sys 0.003s
I20260809 01:45:35.380824 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b870347b07b04de5b833ab4fd3a2475f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:35.382840 11391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b870347b07b04de5b833ab4fd3a2475f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:35.383033 10804 leader_election.cc:343] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:35.385146 11553 tablet_bootstrap.cc:492] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
W20260809 01:45:35.386708 10805 leader_election.cc:343] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:35.387033 11815 leader_election.cc:290] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:35.387287 10805 leader_election.cc:304] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:35.390041 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet da86b5e4a27544c59ae6bb387440d246 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da86b5e4a27544c59ae6bb387440d246 already in progress: creating tablet
W20260809 01:45:35.390166 11063 leader_election.cc:343] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 43d84f310d8e4df7b8b2d602289025ae
I20260809 01:45:35.394865 11624 raft_consensus.cc:2749] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:35.396142 11551 tablet_bootstrap.cc:492] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:35.413705 11554 raft_consensus.cc:359] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.446485 11554 raft_consensus.cc:385] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:35.452622 11554 raft_consensus.cc:740] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
W20260809 01:45:35.461995 10576 catalog_manager.cc:5017] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 825c568f98014fbf92ade7bd99763581 because the tablet was not found. No further retry: Not found: Tablet not found: 825c568f98014fbf92ade7bd99763581
W20260809 01:45:35.462965 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 12fb852b8ea8478cb160a265459865c4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12fb852b8ea8478cb160a265459865c4 already in progress: creating tablet
W20260809 01:45:35.463797 10576 catalog_manager.cc:5017] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet dfb4ad36ca954c7c99407612c0f30711 because the tablet was not found. No further retry: Not found: Tablet not found: dfb4ad36ca954c7c99407612c0f30711
W20260809 01:45:35.464498 10576 catalog_manager.cc:5017] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet a8c4892ed69143cfb9659a6f4a9d9e55 because the tablet was not found. No further retry: Not found: Tablet not found: a8c4892ed69143cfb9659a6f4a9d9e55
W20260809 01:45:35.465154 10576 catalog_manager.cc:5017] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet af8056e8875c42948923e1d76e49d6d0 because the tablet was not found. No further retry: Not found: Tablet not found: af8056e8875c42948923e1d76e49d6d0
I20260809 01:45:35.463699 11554 consensus_queue.cc:260] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.467775 11888 raft_consensus.cc:515] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:35.471179 11815 raft_consensus.cc:493] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:35.471642 11815 raft_consensus.cc:515] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:35.477865 11837 raft_consensus.cc:493] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:35.478542 11837 raft_consensus.cc:515] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:35.482929 11063 leader_election.cc:343] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 61c83b8bcf54461bb24565e3a989100a
I20260809 01:45:35.485936 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0091542071441e5bad0c2dca819f37f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:35.487694 11391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c83b8bcf54461bb24565e3a989100a" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:35.488549 11391 raft_consensus.cc:2468] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:35.489708 11815 leader_election.cc:290] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:35.491873 11554 ts_tablet_manager.cc:1434] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.125s user 0.005s sys 0.007s
I20260809 01:45:35.491518 11062 leader_election.cc:304] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:35.493320 11554 tablet_bootstrap.cc:492] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
W20260809 01:45:35.506304 10592 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 737d7047a0c74a03b3203112987c0cb2 consensus state has no leader
I20260809 01:45:35.508144 10592 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_f860a62234ce42b8ae4c4ff3430df44c" } new_table_name: "default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0" modify_external_catalogs: true
I20260809 01:45:35.512377 11837 leader_election.cc:290] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:35.514297 11391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0091542071441e5bad0c2dca819f37f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:35.516510 10805 leader_election.cc:343] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:35.536698 11888 leader_election.cc:290] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:35.541196 11551 tablet_bootstrap.cc:654] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:35.553716 11554 tablet_bootstrap.cc:654] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:35.563125 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 71760.6us
I20260809 01:45:35.566387 11391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d84f310d8e4df7b8b2d602289025ae" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:35.567070 11391 raft_consensus.cc:3060] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:35.570005 11063 leader_election.cc:343] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 43d84f310d8e4df7b8b2d602289025ae
I20260809 01:45:35.570574 11815 raft_consensus.cc:2804] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:35.570912 11815 raft_consensus.cc:493] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:35.571202 11815 raft_consensus.cc:3060] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:35.596544 11553 tablet_bootstrap.cc:654] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:35.597414 10581 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.533s user 0.016s sys 0.024s
W20260809 01:45:35.597990 10581 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.533s user 0.016s sys 0.024s
W20260809 01:45:35.598419 10581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.125:42063, user_credentials={real_user=slave}} blocked reactor thread for 534093us
I20260809 01:45:35.598968 11815 raft_consensus.cc:515] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:35.604772 11063 leader_election.cc:343] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 61c83b8bcf54461bb24565e3a989100a
I20260809 01:45:35.608316 11919 tablet.cc:1724] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:35.623411 11554 tablet_bootstrap.cc:492] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:35.623818 11554 ts_tablet_manager.cc:1403] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.131s user 0.010s sys 0.010s
I20260809 01:45:35.635620 11554 raft_consensus.cc:359] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.636312 11554 raft_consensus.cc:385] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:35.636682 11554 raft_consensus.cc:740] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:35.613328 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_e121f180daaa4fa9b5d516f74f9b75b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:35.649288 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e121f180daaa4fa9b5d516f74f9b75b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:35.649022 11554 consensus_queue.cc:260] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.658427 11554 ts_tablet_manager.cc:1434] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.034s user 0.005s sys 0.005s
I20260809 01:45:35.659480 11554 tablet_bootstrap.cc:492] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:35.658424 11551 tablet_bootstrap.cc:492] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:35.681723 11551 ts_tablet_manager.cc:1403] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.286s user 0.017s sys 0.007s
I20260809 01:45:35.685894 11892 raft_consensus.cc:493] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:35.686506 11892 raft_consensus.cc:515] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.691797 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b103e2d3854eb4bf8e5cd649cbe129" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:35.692637 11389 raft_consensus.cc:2468] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 0.
I20260809 01:45:35.693598 11892 leader_election.cc:290] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:35.677361 11837 raft_consensus.cc:493] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:35.694772 11837 raft_consensus.cc:515] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.693003 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_5cf8eb3b39ee45f68e7a554f32e35f3d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:35.710208 10805 leader_election.cc:304] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:35.712085 11892 raft_consensus.cc:2804] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:35.712612 11892 raft_consensus.cc:493] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:35.713205 11892 raft_consensus.cc:3060] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:35.714618 11551 raft_consensus.cc:359] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.715328 11551 raft_consensus.cc:385] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:35.715672 11551 raft_consensus.cc:740] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:35.717443 11551 consensus_queue.cc:260] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.719970 10586 master_service.cc:960] ReplaceTablet: received request to replace tablet 637f0d6261f041b5aeff5fcc709a3dfc from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:35.725908 11837 leader_election.cc:290] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:35.728825 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0da8d25290814e66b99c2cc1d5f8b0cf" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:35.730093 11551 ts_tablet_manager.cc:1434] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.048s user 0.008s sys 0.000s
I20260809 01:45:35.731194 11551 tablet_bootstrap.cc:492] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
W20260809 01:45:35.731338 10805 leader_election.cc:343] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:35.735124 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5cf8eb3b39ee45f68e7a554f32e35f3d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:35.754259 11554 tablet_bootstrap.cc:654] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:35.759877 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c83b8bcf54461bb24565e3a989100a" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:35.760443 11389 raft_consensus.cc:3060] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:35.760250 11815 leader_election.cc:290] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:35.755190 10804 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 271980us
I20260809 01:45:35.765360 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b103e2d3854eb4bf8e5cd649cbe129" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:35.766402 10804 leader_election.cc:343] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:35.767179 10804 leader_election.cc:304] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:35.770210 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0da8d25290814e66b99c2cc1d5f8b0cf" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:35.779970 11389 raft_consensus.cc:2468] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:35.774622 11553 tablet_bootstrap.cc:492] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:35.780652 11553 ts_tablet_manager.cc:1403] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.396s user 0.008s sys 0.015s
I20260809 01:45:35.781261 11916 raft_consensus.cc:2749] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:35.781935 11391 raft_consensus.cc:2468] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:35.783239 11062 leader_election.cc:304] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:35.783305 11553 raft_consensus.cc:359] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.784155 11553 raft_consensus.cc:385] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:35.784425 11062 leader_election.cc:304] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:35.784528 11553 raft_consensus.cc:740] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:35.785463 11888 raft_consensus.cc:2804] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:35.785946 11888 raft_consensus.cc:697] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:35.785480 11553 consensus_queue.cc:260] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:35.786559 10804 leader_election.cc:343] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:35.786767 11888 consensus_queue.cc:237] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:35.787448 10804 leader_election.cc:343] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:35.787796 10804 leader_election.cc:304] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:35.789170 11886 raft_consensus.cc:2804] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:35.778393 11892 raft_consensus.cc:515] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.788487 11916 raft_consensus.cc:2749] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:35.812180 11892 leader_election.cc:290] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:35.801806 11553 ts_tablet_manager.cc:1434] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.021s user 0.008s sys 0.000s
I20260809 01:45:35.818667 11553 tablet_bootstrap.cc:492] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:35.820654 11391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b103e2d3854eb4bf8e5cd649cbe129" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:35.822651 11391 raft_consensus.cc:3060] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:35.832221 11554 tablet_bootstrap.cc:492] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:35.832684 11554 ts_tablet_manager.cc:1403] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.173s user 0.010s sys 0.027s
I20260809 01:45:35.835608 11553 tablet_bootstrap.cc:654] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:35.835244 11554 raft_consensus.cc:359] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.835835 11554 raft_consensus.cc:385] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:35.836089 11554 raft_consensus.cc:740] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:35.836974 11554 consensus_queue.cc:260] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.848414 11554 ts_tablet_manager.cc:1434] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
I20260809 01:45:35.856616 11554 tablet_bootstrap.cc:492] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:35.859341 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b103e2d3854eb4bf8e5cd649cbe129" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:35.860728 11551 tablet_bootstrap.cc:654] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:35.863477 11554 tablet_bootstrap.cc:654] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:35.878873 10804 leader_election.cc:343] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:35.880306 11554 tablet_bootstrap.cc:492] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:35.880772 11554 ts_tablet_manager.cc:1403] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.024s user 0.014s sys 0.005s
I20260809 01:45:35.885310 11554 raft_consensus.cc:359] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.886126 11391 raft_consensus.cc:2468] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 1.
I20260809 01:45:35.886304 11554 raft_consensus.cc:385] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:35.886570 11554 raft_consensus.cc:740] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:35.887478 10805 leader_election.cc:304] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:35.888273 11892 raft_consensus.cc:2804] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:35.888715 11892 raft_consensus.cc:697] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Becoming Leader. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Running, Role: LEADER
I20260809 01:45:35.889971 11892 consensus_queue.cc:237] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.906924 11910 raft_consensus.cc:493] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:35.907591 11553 tablet_bootstrap.cc:492] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:35.907042 11886 raft_consensus.cc:697] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:35.907601 11910 raft_consensus.cc:515] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.908030 11553 ts_tablet_manager.cc:1403] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.090s user 0.007s sys 0.006s
I20260809 01:45:35.888001 11554 consensus_queue.cc:260] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.912652 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ead75d766d0346f6823074d90ba8054e" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:35.913539 11156 raft_consensus.cc:2468] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:35.919601 11551 tablet_bootstrap.cc:492] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:35.919988 11554 ts_tablet_manager.cc:1434] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.039s user 0.004s sys 0.003s
I20260809 01:45:35.920099 11551 ts_tablet_manager.cc:1403] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.189s user 0.010s sys 0.005s
I20260809 01:45:35.920971 11554 tablet_bootstrap.cc:492] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:35.923190 11553 raft_consensus.cc:359] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.924022 11553 raft_consensus.cc:385] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:35.924379 11553 raft_consensus.cc:740] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:35.925225 11553 consensus_queue.cc:260] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.931723 11916 raft_consensus.cc:493] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:35.909564 11886 consensus_queue.cc:237] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:35.924037 11551 raft_consensus.cc:359] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.942262 11551 raft_consensus.cc:385] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:35.942410 11553 ts_tablet_manager.cc:1434] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.034s user 0.004s sys 0.004s
I20260809 01:45:35.942761 11283 leader_election.cc:304] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:35.946516 11655 raft_consensus.cc:2804] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:35.947059 11655 raft_consensus.cc:493] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:35.947595 11655 raft_consensus.cc:3060] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:35.948474 11551 raft_consensus.cc:740] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:35.949750 11551 consensus_queue.cc:260] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.953675 11553 tablet_bootstrap.cc:492] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:35.932467 11916 raft_consensus.cc:515] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:35.956436 11551 ts_tablet_manager.cc:1434] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.036s user 0.001s sys 0.005s
I20260809 01:45:35.957223 11551 tablet_bootstrap.cc:492] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:35.960058 11554 tablet_bootstrap.cc:654] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:35.960320 11916 leader_election.cc:290] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:35.962895 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12fb852b8ea8478cb160a265459865c4" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:35.965207 10804 leader_election.cc:343] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:35.974921 10597 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 5f8824953c794450ba274b160d891fcb (table default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]). Expected version 1 got 0
I20260809 01:45:35.976177 10597 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 44ac6aecc93447f7935fbd656fd2a86b (table default.Table_57234dd818414b7397d1ac621bdb06a7 [id=17cbc8bf77e142d08d03bd7a051e00ad]). Expected version 2 got 0
I20260809 01:45:35.977823 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 9b51f999fab24b1ca7451ff543a124a4 (DEFAULT_TABLE table=default.Table_5264c640dfc04d9bae010805cdaf4ea6 [id=c7b50f12c7934944aceb691a5d7b0609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:35.978832 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b51f999fab24b1ca7451ff543a124a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:35.978863 11553 tablet_bootstrap.cc:654] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:35.988380 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 9b51f999fab24b1ca7451ff543a124a4 (DEFAULT_TABLE table=default.Table_5264c640dfc04d9bae010805cdaf4ea6 [id=c7b50f12c7934944aceb691a5d7b0609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:35.989749 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b51f999fab24b1ca7451ff543a124a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:35.989485 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 9b51f999fab24b1ca7451ff543a124a4 (DEFAULT_TABLE table=default.Table_5264c640dfc04d9bae010805cdaf4ea6 [id=c7b50f12c7934944aceb691a5d7b0609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:35.990466 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b51f999fab24b1ca7451ff543a124a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.000816 11888 raft_consensus.cc:493] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:36.001487 11888 raft_consensus.cc:515] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.002991 11910 leader_election.cc:290] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:36.003283 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ead75d766d0346f6823074d90ba8054e" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:36.004086 10905 raft_consensus.cc:2468] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:36.004698 11553 tablet_bootstrap.cc:492] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:36.005064 11553 ts_tablet_manager.cc:1403] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.052s user 0.008s sys 0.006s
I20260809 01:45:36.009011 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bee541fc1345fa83ffebeba05d8c8e" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:36.009479 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bee541fc1345fa83ffebeba05d8c8e" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:36.009943 11554 tablet_bootstrap.cc:492] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:36.010066 10905 raft_consensus.cc:2468] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:36.010524 11554 ts_tablet_manager.cc:1403] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.090s user 0.015s sys 0.003s
W20260809 01:45:36.011685 11062 leader_election.cc:343] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.012539 11888 leader_election.cc:290] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:36.013487 11391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12fb852b8ea8478cb160a265459865c4" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:36.014991 10805 leader_election.cc:343] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.015528 10805 leader_election.cc:304] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:36.015869 11554 raft_consensus.cc:359] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.016499 11554 raft_consensus.cc:385] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.016727 11554 raft_consensus.cc:740] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
W20260809 01:45:36.019217 10581 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 60d356fcc48441ca80f2df29b533404f (table default.Table_29a4bd6bd10a473dbd9dbda008b9eb16 [id=67ee87fabc7749279341528fbcce16d4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 60d356fcc48441ca80f2df29b533404f consensus state has no leader
I20260809 01:45:36.022099 11369 tablet_service.cc:1511] Processing CreateTablet for tablet 5af35cf31a66474cb0575b2fb1547616 (DEFAULT_TABLE table=default.Table_8031d30fd4be45869a2e535707f3459b [id=ab949c442b89438cbd00a59630ab23e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.023119 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5af35cf31a66474cb0575b2fb1547616. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.023651 10586 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 637f0d6261f041b5aeff5fcc709a3dfc
I20260809 01:45:36.021209 11554 consensus_queue.cc:260] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.026252 10586 catalog_manager.cc:6807] ReplaceTablet: tablet 637f0d6261f041b5aeff5fcc709a3dfc deleted and replaced by tablet 995a37a9e6b14889b876da5d76c89b84
I20260809 01:45:36.027292 10586 master_service.cc:960] ReplaceTablet: received request to replace tablet 7501d985958746bea58e0ff752d7fed8 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:36.012383 11063 leader_election.cc:304] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:36.027874 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet 637f0d6261f041b5aeff5fcc709a3dfc with delete_type TABLET_DATA_DELETED (replaced by tablet 995a37a9e6b14889b876da5d76c89b84) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:36.016741 11123 tablet_service.cc:1511] Processing CreateTablet for tablet 5af35cf31a66474cb0575b2fb1547616 (DEFAULT_TABLE table=default.Table_8031d30fd4be45869a2e535707f3459b [id=ab949c442b89438cbd00a59630ab23e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.029316 11888 raft_consensus.cc:2804] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:36.029670 11123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5af35cf31a66474cb0575b2fb1547616. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.029882 11888 raft_consensus.cc:493] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:36.031318 11888 raft_consensus.cc:3060] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:36.026512 11129 tablet_service.cc:1558] Processing DeleteTablet for tablet 637f0d6261f041b5aeff5fcc709a3dfc with delete_type TABLET_DATA_DELETED (replaced by tablet 995a37a9e6b14889b876da5d76c89b84) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:36.036492 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet 637f0d6261f041b5aeff5fcc709a3dfc with delete_type TABLET_DATA_DELETED (replaced by tablet 995a37a9e6b14889b876da5d76c89b84) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:36.040500 11553 raft_consensus.cc:359] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.041200 11553 raft_consensus.cc:385] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.047667 11554 ts_tablet_manager.cc:1434] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.037s user 0.007s sys 0.000s
I20260809 01:45:36.040500 11916 raft_consensus.cc:2749] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:36.041494 11553 raft_consensus.cc:740] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:36.042539 11892 raft_consensus.cc:493] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:36.049355 11892 raft_consensus.cc:515] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.050904 11554 tablet_bootstrap.cc:492] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:36.052241 11892 leader_election.cc:290] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:36.049449 11553 consensus_queue.cc:260] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.055256 11553 ts_tablet_manager.cc:1434] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.050s user 0.004s sys 0.004s
I20260809 01:45:36.043757 10598 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 5f8824953c794450ba274b160d891fcb (table default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]). Expected version 1 got 0
I20260809 01:45:36.057049 11553 tablet_bootstrap.cc:492] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
W20260809 01:45:36.045913 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 637f0d6261f041b5aeff5fcc709a3dfc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 637f0d6261f041b5aeff5fcc709a3dfc already in progress: creating tablet
I20260809 01:45:36.058432 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c795f81d2bc248eeb39b64d6aaa0370c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:36.060564 10804 leader_election.cc:343] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.065339 10596 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 5f8824953c794450ba274b160d891fcb (table default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]). Expected version 1 got 0
I20260809 01:45:36.068074 10596 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233]). Expected version 2 got 0
I20260809 01:45:36.069425 11391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c795f81d2bc248eeb39b64d6aaa0370c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:36.070967 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 48b7459ad0974cb9948d647459b22260
I20260809 01:45:36.072103 11553 tablet_bootstrap.cc:654] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:36.074841 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet 48b7459ad0974cb9948d647459b22260 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:35 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:36.075893 11949 tablet_replica.cc:333] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:36.076545 10589 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_837bb268533c42268f543bcc9732f18e" } new_table_name: "default.Table_834b6f679f5a401c9a32de707d5b931d" modify_external_catalogs: true
I20260809 01:45:36.077262 11949 raft_consensus.cc:2243] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:36.078193 11949 raft_consensus.cc:2272] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:36.078655 11120 tablet_service.cc:1558] Processing DeleteTablet for tablet 48b7459ad0974cb9948d647459b22260 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:35 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:36.081691 11949 ts_tablet_manager.cc:1916] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:36.095036 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 48b7459ad0974cb9948d647459b22260 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:35 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:36.095460 11365 tablet_service.cc:1558] Processing DeleteTablet for tablet e155d16b1d594a789f539390a44bc39f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:32 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:36.074550 11554 tablet_bootstrap.cc:654] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:36.097334 10805 leader_election.cc:343] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.098440 10805 leader_election.cc:304] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:36.100821 11365 tablet_service.cc:1558] Processing DeleteTablet for tablet 2674c9dbf9e44d63b41427f2fda4e311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:29 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:36.105417 11120 tablet_service.cc:1511] Processing CreateTablet for tablet cef88ffe9e95437da81793795742def3 (DEFAULT_TABLE table=default.Table_e53beaa5c2fe49e3ba6eaa8181b5106a [id=40cb4403a8a94bd3938983441e9254d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.105693 10874 tablet_service.cc:1511] Processing CreateTablet for tablet 5af35cf31a66474cb0575b2fb1547616 (DEFAULT_TABLE table=default.Table_8031d30fd4be45869a2e535707f3459b [id=ab949c442b89438cbd00a59630ab23e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.107117 11120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cef88ffe9e95437da81793795742def3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.103920 11916 raft_consensus.cc:2749] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:36.110850 11910 raft_consensus.cc:493] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:36.111274 11910 raft_consensus.cc:515] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.112571 10870 tablet_service.cc:1511] Processing CreateTablet for tablet cef88ffe9e95437da81793795742def3 (DEFAULT_TABLE table=default.Table_e53beaa5c2fe49e3ba6eaa8181b5106a [id=40cb4403a8a94bd3938983441e9254d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.115356 11910 leader_election.cc:290] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:36.115968 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bee541fc1345fa83ffebeba05d8c8e" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:36.120021 10874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5af35cf31a66474cb0575b2fb1547616. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.120620 11365 tablet_service.cc:1511] Processing CreateTablet for tablet cef88ffe9e95437da81793795742def3 (DEFAULT_TABLE table=default.Table_e53beaa5c2fe49e3ba6eaa8181b5106a [id=40cb4403a8a94bd3938983441e9254d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.121776 11365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cef88ffe9e95437da81793795742def3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.122970 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cef88ffe9e95437da81793795742def3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.102402 11892 raft_consensus.cc:493] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:36.124665 11892 raft_consensus.cc:515] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.127717 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aac67e7819f4a70b1050b558868f27f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:36.129119 10804 leader_election.cc:343] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.137995 11124 tablet_service.cc:1511] Processing CreateTablet for tablet dc230f25baf94cbf80aa8e2771ec23ba (DEFAULT_TABLE table=default.Table_1998ec81597a4c4584dc4e4f90caa274 [id=cb347b72c1cf436095c6043c5b169641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.138978 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc230f25baf94cbf80aa8e2771ec23ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.143425 11553 tablet_bootstrap.cc:492] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:36.143913 11553 ts_tablet_manager.cc:1403] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.087s user 0.011s sys 0.011s
I20260809 01:45:36.145283 11655 raft_consensus.cc:515] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:36.150431 10588 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.67:40463 (request call id 14) took 2239 ms (client timeout 1000 ms). Trace:
I20260809 01:45:36.151296 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ead75d766d0346f6823074d90ba8054e" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:36.151984 11160 raft_consensus.cc:3060] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:36.154752 11553 raft_consensus.cc:359] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.155647 11553 raft_consensus.cc:385] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.155611 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 61c83b8bcf54461bb24565e3a989100a with delete_type TABLET_DATA_DELETED (replaced by tablet e5888017345c48c7bafe6f1c6e6945cc) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:36.155962 11553 raft_consensus.cc:740] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:36.156679 11553 consensus_queue.cc:260] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.160355 11553 ts_tablet_manager.cc:1434] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.016s user 0.008s sys 0.000s
I20260809 01:45:36.161375 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 5f8824953c794450ba274b160d891fcb (table default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]). Expected version 1 got 0
I20260809 01:45:36.162199 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233]). Expected version 2 got 0
I20260809 01:45:36.165817 10597 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_ff0d273341b54d79b4e634ea694e982f [id=46b096a453084568935bc0d5855cb0f7] alter complete (version 3)
I20260809 01:45:36.167548 11553 tablet_bootstrap.cc:492] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:36.170110 10586 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 7501d985958746bea58e0ff752d7fed8
I20260809 01:45:36.171370 11126 tablet_service.cc:1558] Processing DeleteTablet for tablet e155d16b1d594a789f539390a44bc39f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:32 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:36.171448 11125 tablet_service.cc:1558] Processing DeleteTablet for tablet 2674c9dbf9e44d63b41427f2fda4e311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:29 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:36.172549 10586 catalog_manager.cc:6807] ReplaceTablet: tablet 7501d985958746bea58e0ff752d7fed8 deleted and replaced by tablet b2726cb09f734ed18bb9379f94f3f22a
I20260809 01:45:36.173748 11125 tablet_service.cc:1558] Processing DeleteTablet for tablet 7501d985958746bea58e0ff752d7fed8 with delete_type TABLET_DATA_DELETED (replaced by tablet b2726cb09f734ed18bb9379f94f3f22a) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:36.149451 11655 leader_election.cc:290] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:36.176048 10597 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.66:57751 (request call id 22) took 1441 ms (client timeout 1000 ms). Trace:
W20260809 01:45:36.177554 11945 log.cc:927] Time spent T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7: Append to log took a long time: real 0.052s user 0.001s sys 0.000s
W20260809 01:45:36.176478 10597 rpcz_store.cc:269] 0809 01:45:34.735050 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:34.806559 (+ 71509us) service_pool.cc:224] Handling call
0809 01:45:34.898833 (+ 92274us) spinlock_profiling.cc:243] Waited 35 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:36.175959 (+1277126us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:36.044355 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:36.044713 (+ 358us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:36.044737 (+ 24us) write_op.cc:435] Acquired schema lock
0809 01:45:36.044750 (+ 13us) tablet.cc:662] Decoding operations
0809 01:45:36.044997 (+ 247us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:36.045035 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:36.045057 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:36.045187 (+ 130us) tablet.cc:701] Row locks acquired
0809 01:45:36.045200 (+ 13us) write_op.cc:260] PREPARE: finished
0809 01:45:36.045309 (+ 109us) write_op.cc:270] Start()
0809 01:45:36.045415 (+ 106us) write_op.cc:276] Timestamp: P: 1786239936045237 usec, L: 0
0809 01:45:36.045435 (+ 20us) op_driver.cc:348] REPLICATION: starting
0809 01:45:36.046011 (+ 576us) log.cc:844] Serialized 356 byte log entry
0809 01:45:36.061316 (+ 15305us) op_driver.cc:464] REPLICATION: finished
0809 01:45:36.061844 (+ 528us) write_op.cc:301] APPLY: starting
0809 01:45:36.061900 (+ 56us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:36.062032 (+ 132us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:36.062127 (+ 95us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:36.062399 (+ 272us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:36.062587 (+ 188us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:36.062653 (+ 66us) write_op.cc:312] APPLY: finished
0809 01:45:36.062929 (+ 276us) log.cc:844] Serialized 23 byte log entry
0809 01:45:36.063111 (+ 182us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:36.063188 (+ 77us) write_op.cc:454] Released schema lock
0809 01:45:36.063225 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:36.063629 (+ 404us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:36.063676 (+ 47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":19329,"reported_tablets":8,"spinlock_wait_cycles":80531328,"child_traces":[["op",{"apply.queue_time_us":365,"apply.run_cpu_time_us":1655,"apply.run_wall_time_us":1655,"num_ops":1,"prepare.queue_time_us":13323,"prepare.run_cpu_time_us":2072,"prepare.run_wall_time_us":5210,"raft.queue_time_us":167,"raft.run_cpu_time_us":792,"raft.run_wall_time_us":792,"replication_time_us":15722}]]}
I20260809 01:45:36.180886 11888 raft_consensus.cc:515] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.182572 11156 raft_consensus.cc:2393] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f60c9ac9414403d9ea5a97b103c2316 in current term 1: Already voted for candidate 33b5652538e64ed58a09993d3188f8d7 in this term.
I20260809 01:45:36.184310 11125 tablet_service.cc:1558] Processing DeleteTablet for tablet 61c83b8bcf54461bb24565e3a989100a with delete_type TABLET_DATA_DELETED (replaced by tablet e5888017345c48c7bafe6f1c6e6945cc) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:36.186631 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bee541fc1345fa83ffebeba05d8c8e" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:36.187443 10910 raft_consensus.cc:3060] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:36.188174 11949 ts_tablet_manager.cc:1929] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:36.188861 11949 log.cc:1199] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/48b7459ad0974cb9948d647459b22260
I20260809 01:45:36.190161 11949 ts_tablet_manager.cc:1950] T 48b7459ad0974cb9948d647459b22260 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:36.193518 11553 tablet_bootstrap.cc:654] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:36.220474 11160 raft_consensus.cc:2468] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:36.222563 11283 leader_election.cc:304] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:36.223814 11553 tablet_bootstrap.cc:492] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:36.224220 11553 ts_tablet_manager.cc:1403] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.057s user 0.008s sys 0.008s
I20260809 01:45:36.225344 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bee541fc1345fa83ffebeba05d8c8e" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:36.226080 11389 raft_consensus.cc:3060] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:36.227367 11655 raft_consensus.cc:2804] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:36.228116 11655 raft_consensus.cc:697] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:36.229364 11655 consensus_queue.cc:237] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.230202 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 5050a590fe0240ac8c9f9191ec2475a1 (DEFAULT_TABLE table=default.Table_fe1acd943c5c42c9b09a0a5845cbee68 [id=487f10bf79a944168eece2b1e8a2daf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.232200 11553 raft_consensus.cc:359] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.234169 11553 raft_consensus.cc:385] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.234746 11553 raft_consensus.cc:740] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
W20260809 01:45:36.209954 10589 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 36a542c773b7438ead16c5b3af224cf9 has no consensus state
I20260809 01:45:36.235469 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_b92eb95256a74ee195ec93866ec1b673"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:36.237106 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b92eb95256a74ee195ec93866ec1b673 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:36.236057 11553 consensus_queue.cc:260] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:36.238092 10592 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 85d0a9d73d4841c393e7d2cef41fee89 consensus state has no leader
I20260809 01:45:36.239413 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_8a405d19ab94416c8d58fff878edae0a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:36.242555 11553 ts_tablet_manager.cc:1434] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.018s user 0.009s sys 0.003s
W20260809 01:45:36.241214 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a405d19ab94416c8d58fff878edae0a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:36.246215 11553 tablet_bootstrap.cc:492] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:36.229033 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 48b7459ad0974cb9948d647459b22260 (table default.Table_d37c43f0673044eb826f3944101bcd4b [id=ec2be55379204f57a7bf8a8739b73033]) successfully deleted
I20260809 01:45:36.231413 10598 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_57234dd818414b7397d1ac621bdb06a7 [id=17cbc8bf77e142d08d03bd7a051e00ad] alter complete (version 2)
I20260809 01:45:36.251806 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bee541fc1345fa83ffebeba05d8c8e" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:36.253626 11888 leader_election.cc:290] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:36.253950 11289 leader_election.cc:304] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:36.258545 11389 raft_consensus.cc:2468] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:36.259186 11910 raft_consensus.cc:2749] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:36.270375 10869 tablet_service.cc:1511] Processing CreateTablet for tablet dc230f25baf94cbf80aa8e2771ec23ba (DEFAULT_TABLE table=default.Table_1998ec81597a4c4584dc4e4f90caa274 [id=cb347b72c1cf436095c6043c5b169641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.271287 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc230f25baf94cbf80aa8e2771ec23ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.273953 10874 tablet_service.cc:1558] Processing DeleteTablet for tablet 7501d985958746bea58e0ff752d7fed8 with delete_type TABLET_DATA_DELETED (replaced by tablet b2726cb09f734ed18bb9379f94f3f22a) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:36.284001 11062 leader_election.cc:304] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:36.285682 11888 raft_consensus.cc:2804] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:36.287237 11888 raft_consensus.cc:697] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:36.288620 11888 consensus_queue.cc:237] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.299737 11553 tablet_bootstrap.cc:654] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:36.302227 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ead75d766d0346f6823074d90ba8054e" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:36.303013 10905 raft_consensus.cc:3060] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:36.315148 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5050a590fe0240ac8c9f9191ec2475a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.317446 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 7501d985958746bea58e0ff752d7fed8 with delete_type TABLET_DATA_DELETED (replaced by tablet b2726cb09f734ed18bb9379f94f3f22a) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:36.319355 11353 tablet_service.cc:1511] Processing CreateTablet for tablet dc230f25baf94cbf80aa8e2771ec23ba (DEFAULT_TABLE table=default.Table_1998ec81597a4c4584dc4e4f90caa274 [id=cb347b72c1cf436095c6043c5b169641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.322252 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc230f25baf94cbf80aa8e2771ec23ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.331799 11553 tablet_bootstrap.cc:492] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:36.332305 11553 ts_tablet_manager.cc:1403] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.086s user 0.010s sys 0.009s
I20260809 01:45:36.339658 11553 raft_consensus.cc:359] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.340425 10910 raft_consensus.cc:2468] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:36.340444 11553 raft_consensus.cc:385] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.348479 11553 raft_consensus.cc:740] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:36.353425 11553 consensus_queue.cc:260] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:36.353693 10580 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 124667us
I20260809 01:45:36.355800 10595 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 98bee541fc1345fa83ffebeba05d8c8e
I20260809 01:45:36.356184 11553 ts_tablet_manager.cc:1434] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.024s user 0.003s sys 0.007s
I20260809 01:45:36.358566 10595 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 118) took 1232 ms. Trace:
I20260809 01:45:36.362144 11553 tablet_bootstrap.cc:492] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
W20260809 01:45:36.363345 10599 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 24) took 2101 ms (client timeout 1000 ms). Trace:
I20260809 01:45:36.363829 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aac67e7819f4a70b1050b558868f27f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:36.367954 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_5f68b1cdb1a347f78466f3333fd113ff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:36.373396 11551 tablet_bootstrap.cc:654] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:36.344228 11892 leader_election.cc:290] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:36.378894 10598 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.67:40463 (request call id 15) took 1229 ms (client timeout 1000 ms). Trace:
I20260809 01:45:36.381016 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 5050a590fe0240ac8c9f9191ec2475a1 (DEFAULT_TABLE table=default.Table_fe1acd943c5c42c9b09a0a5845cbee68 [id=487f10bf79a944168eece2b1e8a2daf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.382788 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5050a590fe0240ac8c9f9191ec2475a1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:36.383659 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f68b1cdb1a347f78466f3333fd113ff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:36.363727 10599 rpcz_store.cc:269] 0809 01:45:34.262484 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:34.287880 (+ 25396us) service_pool.cc:224] Handling call
0809 01:45:36.363252 (+2075372us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:34.884479 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:34.884834 (+ 355us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:34.884856 (+ 22us) write_op.cc:435] Acquired schema lock
0809 01:45:34.884868 (+ 12us) tablet.cc:662] Decoding operations
0809 01:45:34.885116 (+ 248us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:34.885146 (+ 30us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:34.885168 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:34.920399 (+ 35231us) tablet.cc:701] Row locks acquired
0809 01:45:34.920414 (+ 15us) write_op.cc:260] PREPARE: finished
0809 01:45:34.920512 (+ 98us) write_op.cc:270] Start()
0809 01:45:34.920601 (+ 89us) write_op.cc:276] Timestamp: P: 1786239934920459 usec, L: 0
0809 01:45:34.920616 (+ 15us) op_driver.cc:348] REPLICATION: starting
0809 01:45:34.950046 (+ 29430us) log.cc:844] Serialized 527 byte log entry
0809 01:45:34.958144 (+ 8098us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678d951 00007fa2c6820208 00007fa2c6caff61 00007fa2c6caf738 00007fa2c6cb43d1 00007fa2c6cb4389 00007fa2c6cb4319 00007fa2c6cb42e1 00007fa2c6cb350d 00000000005bd8c4 00000000005bd6f9
0809 01:45:35.003817 (+ 45673us) op_driver.cc:464] REPLICATION: finished
0809 01:45:35.013839 (+ 10022us) write_op.cc:301] APPLY: starting
0809 01:45:35.013900 (+ 61us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:35.013997 (+ 97us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:35.014022 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:35.014186 (+ 164us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:35.014322 (+ 136us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:35.014367 (+ 45us) write_op.cc:312] APPLY: finished
0809 01:45:35.014532 (+ 165us) log.cc:844] Serialized 23 byte log entry
0809 01:45:35.225854 (+211322us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:35.225940 (+ 86us) write_op.cc:454] Released schema lock
0809 01:45:35.225997 (+ 57us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:35.308226 (+ 82229us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:35.308260 (+ 34us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":9407,"apply.run_cpu_time_us":1493,"apply.run_wall_time_us":294284,"num_ops":1,"prepare.queue_time_us":195183,"prepare.run_cpu_time_us":2821,"prepare.run_wall_time_us":85332,"raft.queue_time_us":681,"raft.run_cpu_time_us":611,"raft.run_wall_time_us":611,"replication_time_us":83099,"spinlock_wait_cycles":66535168}]]}
I20260809 01:45:36.393775 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_198b5459279b480eb8a7b2d0ab6f8c87"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:36.395553 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_198b5459279b480eb8a7b2d0ab6f8c87 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:36.404098 11951 tablet_replica.cc:333] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
W20260809 01:45:36.378646 10805 leader_election.cc:343] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.420724 10805 leader_election.cc:304] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:36.383642 11553 tablet_bootstrap.cc:654] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:36.433552 11124 tablet_service.cc:1558] Processing DeleteTablet for tablet 98bee541fc1345fa83ffebeba05d8c8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:35 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:36.435166 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 6d35f291c92a45899eed0f2f10a3a1d8 (DEFAULT_TABLE table=default.Table_14613c38aa0e4b258598254918b193bb [id=fa2e19a3042b41959165ca54e46ca690]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.437047 11960 raft_consensus.cc:2749] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:36.440861 11123 tablet_service.cc:1511] Processing CreateTablet for tablet c16897857df24c6da5744a99d5f687c0 (DEFAULT_TABLE table=default.Table_6d4e0b3afc864450bdf1a8fb35f7dc75 [id=9b307bc98a124b07a2f46fafb20ce1f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.441682 11554 tablet_bootstrap.cc:492] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
W20260809 01:45:36.441641 10804 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 161138us
I20260809 01:45:36.442098 11554 ts_tablet_manager.cc:1403] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.391s user 0.057s sys 0.051s
I20260809 01:45:36.445811 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet 98bee541fc1345fa83ffebeba05d8c8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:35 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:36.445850 11554 raft_consensus.cc:359] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.446763 11554 raft_consensus.cc:385] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.447011 11554 raft_consensus.cc:740] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:36.447335 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 5050a590fe0240ac8c9f9191ec2475a1 (DEFAULT_TABLE table=default.Table_fe1acd943c5c42c9b09a0a5845cbee68 [id=487f10bf79a944168eece2b1e8a2daf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.447953 11554 consensus_queue.cc:260] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:36.450107 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 7501d985958746bea58e0ff752d7fed8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7501d985958746bea58e0ff752d7fed8 already in progress: creating tablet
I20260809 01:45:36.451128 11554 ts_tablet_manager.cc:1434] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260809 01:45:36.452126 11554 tablet_bootstrap.cc:492] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:36.453123 11910 raft_consensus.cc:493] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:36.453863 11910 raft_consensus.cc:515] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.458462 11910 leader_election.cc:290] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:36.459199 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 6d35f291c92a45899eed0f2f10a3a1d8 (DEFAULT_TABLE table=default.Table_14613c38aa0e4b258598254918b193bb [id=fa2e19a3042b41959165ca54e46ca690]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.459826 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5050a590fe0240ac8c9f9191ec2475a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.461454 10874 tablet_service.cc:1511] Processing CreateTablet for tablet c16897857df24c6da5744a99d5f687c0 (DEFAULT_TABLE table=default.Table_6d4e0b3afc864450bdf1a8fb35f7dc75 [id=9b307bc98a124b07a2f46fafb20ce1f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.463660 10874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16897857df24c6da5744a99d5f687c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.466378 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0da8d25290814e66b99c2cc1d5f8b0cf" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:36.467015 10910 raft_consensus.cc:2468] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:36.468116 11950 tablet_replica.cc:333] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:36.469314 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d35f291c92a45899eed0f2f10a3a1d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.470561 11949 tablet_replica.cc:333] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:36.471802 11949 raft_consensus.cc:2243] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:36.472467 11949 raft_consensus.cc:2272] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:36.474452 11123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16897857df24c6da5744a99d5f687c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.475296 11949 ts_tablet_manager.cc:1916] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:36.478675 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0da8d25290814e66b99c2cc1d5f8b0cf" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:36.480638 11283 leader_election.cc:343] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.486965 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d35f291c92a45899eed0f2f10a3a1d8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:36.487470 10581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.125:42063, user_credentials={real_user=slave}} blocked reactor thread for 45706.8us
I20260809 01:45:36.491155 11289 leader_election.cc:304] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:36.492380 11910 raft_consensus.cc:2804] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:36.492712 11910 raft_consensus.cc:493] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:36.493023 11910 raft_consensus.cc:3060] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:36.494753 10871 tablet_service.cc:1511] Processing CreateTablet for tablet c2f86bba4bd94fbcacdc9c139e9de2d7 (DEFAULT_TABLE table=default.Table_5f4d617a86c2487fa9f64cb58f14de3c [id=f64daa5a2706436081e9166a5f3a0cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.495522 10905 raft_consensus.cc:2468] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:36.496304 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f86bba4bd94fbcacdc9c139e9de2d7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:36.497143 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 637f0d6261f041b5aeff5fcc709a3dfc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 637f0d6261f041b5aeff5fcc709a3dfc already in progress: creating tablet
I20260809 01:45:36.498849 11553 tablet_bootstrap.cc:492] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:36.499286 11553 ts_tablet_manager.cc:1403] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.137s user 0.010s sys 0.014s
I20260809 01:45:36.501601 11553 raft_consensus.cc:359] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.502197 11553 raft_consensus.cc:385] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.502475 11553 raft_consensus.cc:740] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:36.503149 11553 consensus_queue.cc:260] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.504971 11553 ts_tablet_manager.cc:1434] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 01:45:36.507082 10586 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 5f8824953c794450ba274b160d891fcb (table default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]). Expected version 1 got 0
I20260809 01:45:36.509021 11553 tablet_bootstrap.cc:492] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:36.533916 10873 tablet_service.cc:1511] Processing CreateTablet for tablet 7706838afc9f4dbebc3213b82c4da126 (DEFAULT_TABLE table=default.Table_57ef64be310d4250b77b141860e944d5 [id=212edfae1bf849c3b46674ec191ee76d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.539320 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7706838afc9f4dbebc3213b82c4da126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.541682 11553 tablet_bootstrap.cc:654] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:36.379257 10598 rpcz_store.cc:269] 0809 01:45:35.149470 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:35.149663 (+ 193us) service_pool.cc:224] Handling call
0809 01:45:36.231078 (+1081415us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:36.231110 (+ 32us) write_op.cc:454] Released schema lock
0809 01:45:36.378818 (+147708us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:36.184972 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:36.185372 (+ 400us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:36.185395 (+ 23us) write_op.cc:435] Acquired schema lock
0809 01:45:36.185407 (+ 12us) tablet.cc:662] Decoding operations
0809 01:45:36.185670 (+ 263us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:36.185704 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:36.185724 (+ 20us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:36.185875 (+ 151us) tablet.cc:701] Row locks acquired
0809 01:45:36.185891 (+ 16us) write_op.cc:260] PREPARE: finished
0809 01:45:36.185995 (+ 104us) write_op.cc:270] Start()
0809 01:45:36.186078 (+ 83us) write_op.cc:276] Timestamp: P: 1786239936185923 usec, L: 0
0809 01:45:36.186091 (+ 13us) op_driver.cc:348] REPLICATION: starting
0809 01:45:36.189015 (+ 2924us) log.cc:844] Serialized 356 byte log entry
0809 01:45:36.215188 (+ 26173us) op_driver.cc:464] REPLICATION: finished
0809 01:45:36.215607 (+ 419us) write_op.cc:301] APPLY: starting
0809 01:45:36.215674 (+ 67us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:36.215783 (+ 109us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:36.215885 (+ 102us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:36.216065 (+ 180us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:36.216188 (+ 123us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:36.216243 (+ 55us) write_op.cc:312] APPLY: finished
0809 01:45:36.216421 (+ 178us) log.cc:844] Serialized 23 byte log entry
0809 01:45:36.216565 (+ 144us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:36.216633 (+ 68us) write_op.cc:454] Released schema lock
0809 01:45:36.216672 (+ 39us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":280,"apply.run_cpu_time_us":1247,"apply.run_wall_time_us":1242,"num_ops":1,"prepare.queue_time_us":103,"prepare.run_cpu_time_us":1991,"prepare.run_wall_time_us":4498,"raft.queue_time_us":82,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":13,"replication_time_us":29023}]]}
I20260809 01:45:36.554831 10598 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_e121f180daaa4fa9b5d516f74f9b75b3" } modify_external_catalogs: true
I20260809 01:45:36.555965 10598 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_e121f180daaa4fa9b5d516f74f9b75b3" } modify_external_catalogs: true
I20260809 01:45:36.358953 10595 rpcz_store.cc:276] 0809 01:45:35.125791 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:35.126486 (+ 695us) service_pool.cc:224] Handling call
0809 01:45:35.127952 (+ 1466us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:35.128706 (+ 754us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:35.128891 (+ 185us) catalog_manager.cc:2850] Locking tablets
0809 01:45:35.228360 (+ 99469us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:35.297568 (+ 69208us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:35.298090 (+ 522us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:36.355663 (+1057573us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:36.358467 (+ 2804us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:35.246331 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:35.246803 (+ 472us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:35.246839 (+ 36us) write_op.cc:435] Acquired schema lock
0809 01:45:35.246851 (+ 12us) tablet.cc:662] Decoding operations
0809 01:45:35.247227 (+ 376us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:35.247277 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:35.247309 (+ 32us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:35.247669 (+ 360us) tablet.cc:701] Row locks acquired
0809 01:45:35.247701 (+ 32us) write_op.cc:260] PREPARE: finished
0809 01:45:35.247828 (+ 127us) write_op.cc:270] Start()
0809 01:45:35.247906 (+ 78us) write_op.cc:276] Timestamp: P: 1786239935247741 usec, L: 0
0809 01:45:35.247920 (+ 14us) op_driver.cc:348] REPLICATION: starting
0809 01:45:35.248464 (+ 544us) log.cc:844] Serialized 756 byte log entry
0809 01:45:35.293860 (+ 45396us) op_driver.cc:464] REPLICATION: finished
0809 01:45:35.294158 (+ 298us) write_op.cc:301] APPLY: starting
0809 01:45:35.294245 (+ 87us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:35.294389 (+ 144us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:35.294560 (+ 171us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:35.294940 (+ 380us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:35.295207 (+ 267us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:35.295326 (+ 119us) write_op.cc:312] APPLY: finished
0809 01:45:35.295649 (+ 323us) log.cc:844] Serialized 29 byte log entry
0809 01:45:35.295928 (+ 279us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:35.296091 (+ 163us) write_op.cc:454] Released schema lock
0809 01:45:35.296182 (+ 91us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:35.317147 (+ 20965us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b5400030300. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c6be3b41 00007fa2c6cb8d1d 00007fa2c6cb1da2 00007fa2c6cb1533 00007fa2c6cb5741 00007fa2c6cb56f9 00007fa2c6cb5689 00007fa2c6cb5651 00007fa2c6cb487d 00000000005bd8c4 00000000005bd6f9
0809 01:45:35.317351 (+ 204us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:35.317387 (+ 36us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":79,"apply.run_cpu_time_us":5286,"apply.run_wall_time_us":23114,"num_ops":2,"prepare.queue_time_us":9197,"prepare.run_cpu_time_us":2690,"prepare.run_wall_time_us":4689,"raft.queue_time_us":262,"raft.run_cpu_time_us":1337,"raft.run_wall_time_us":24654,"replication_time_us":45842,"spinlock_wait_cycles":27790720}]]}
W20260809 01:45:36.432472 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 637f0d6261f041b5aeff5fcc709a3dfc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 637f0d6261f041b5aeff5fcc709a3dfc already in progress: creating tablet
W20260809 01:45:36.560827 10576 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 128379us
I20260809 01:45:36.561841 11553 tablet_bootstrap.cc:492] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:36.561885 11950 raft_consensus.cc:2243] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:36.562386 11553 ts_tablet_manager.cc:1403] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.054s user 0.009s sys 0.003s
I20260809 01:45:36.562709 11950 raft_consensus.cc:2272] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:36.490157 11951 raft_consensus.cc:2243] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:36.565361 11553 raft_consensus.cc:359] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.566296 11553 raft_consensus.cc:385] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.566644 11553 raft_consensus.cc:740] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:36.566206 10595 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_54419467eb324360916beed8ea56f0db" } new_table_name: "default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4" modify_external_catalogs: true
I20260809 01:45:36.566414 11951 raft_consensus.cc:2272] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:36.567323 11553 consensus_queue.cc:260] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.569380 11553 ts_tablet_manager.cc:1434] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 01:45:36.574249 11553 tablet_bootstrap.cc:492] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:36.576798 10592 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_080ef38220da4f5f8f016c1f733ee2ea" } new_table_name: "default.Table_a5f25d7531bb41e5abbe7be00eb9833a" modify_external_catalogs: true
I20260809 01:45:36.581388 11125 tablet_service.cc:1511] Processing CreateTablet for tablet 7706838afc9f4dbebc3213b82c4da126 (DEFAULT_TABLE table=default.Table_57ef64be310d4250b77b141860e944d5 [id=212edfae1bf849c3b46674ec191ee76d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.583371 11125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7706838afc9f4dbebc3213b82c4da126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.587618 11624 raft_consensus.cc:493] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:36.588418 11624 raft_consensus.cc:515] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.592036 11624 leader_election.cc:290] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:36.593544 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501d985958746bea58e0ff752d7fed8" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:36.594388 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501d985958746bea58e0ff752d7fed8" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:36.601296 10805 leader_election.cc:343] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.604652 11553 tablet_bootstrap.cc:654] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:36.608551 10804 leader_election.cc:343] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.608928 10804 leader_election.cc:304] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:36.609912 11624 raft_consensus.cc:2749] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:36.615593 11129 tablet_service.cc:1511] Processing CreateTablet for tablet c2f86bba4bd94fbcacdc9c139e9de2d7 (DEFAULT_TABLE table=default.Table_5f4d617a86c2487fa9f64cb58f14de3c [id=f64daa5a2706436081e9166a5f3a0cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.626225 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f86bba4bd94fbcacdc9c139e9de2d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.627892 11970 raft_consensus.cc:493] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:36.636451 11553 tablet_bootstrap.cc:492] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:36.636931 11553 ts_tablet_manager.cc:1403] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.063s user 0.002s sys 0.009s
W20260809 01:45:36.150892 10588 rpcz_store.cc:269] 0809 01:45:33.911325 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:33.911482 (+ 157us) service_pool.cc:224] Handling call
0809 01:45:35.314380 (+1402898us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b5400030300. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c6be3b41 00007fa2c6cb8569 00007fa2c6cae18d 00007fa2c6c6f7eb 00007fa2c6c6f5c4 00007fa2c86ccb51 00007fa2c86cd3af 00007fa2c858dc5b 00007fa2c869453e 00007fa2c6241fb4 00007fa2c6241f42
0809 01:45:36.150363 (+835983us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:35.322191 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:35.322655 (+ 464us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:35.322681 (+ 26us) write_op.cc:435] Acquired schema lock
0809 01:45:35.322696 (+ 15us) tablet.cc:662] Decoding operations
0809 01:45:35.322969 (+ 273us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:35.323011 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:35.323036 (+ 25us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:35.323216 (+ 180us) tablet.cc:701] Row locks acquired
0809 01:45:35.323231 (+ 15us) write_op.cc:260] PREPARE: finished
0809 01:45:35.324399 (+ 1168us) write_op.cc:270] Start()
0809 01:45:35.324493 (+ 94us) write_op.cc:276] Timestamp: P: 1786239935324344 usec, L: 0
0809 01:45:35.324511 (+ 18us) op_driver.cc:348] REPLICATION: starting
0809 01:45:35.325026 (+ 515us) log.cc:844] Serialized 527 byte log entry
0809 01:45:35.412371 (+ 87345us) op_driver.cc:464] REPLICATION: finished
0809 01:45:35.412753 (+ 382us) write_op.cc:301] APPLY: starting
0809 01:45:35.412860 (+ 107us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:35.413003 (+ 143us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:35.419167 (+ 6164us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:35.427217 (+ 8050us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:35.427449 (+ 232us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:35.427493 (+ 44us) write_op.cc:312] APPLY: finished
0809 01:45:35.427771 (+ 278us) log.cc:844] Serialized 23 byte log entry
0809 01:45:35.428069 (+ 298us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:35.428217 (+ 148us) write_op.cc:454] Released schema lock
0809 01:45:35.428271 (+ 54us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:35.428616 (+ 345us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:35.428678 (+ 62us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":53453,"reported_tablets":8,"spinlock_wait_cycles":62124416,"child_traces":[["op",{"apply.queue_time_us":143,"apply.run_cpu_time_us":2096,"apply.run_wall_time_us":15822,"num_ops":1,"prepare.queue_time_us":7549,"prepare.run_cpu_time_us":2124,"prepare.run_wall_time_us":4778,"raft.queue_time_us":27259,"raft.run_cpu_time_us":874,"raft.run_wall_time_us":874,"replication_time_us":87734}]]}
I20260809 01:45:36.640223 11553 raft_consensus.cc:359] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:36.640903 11553 raft_consensus.cc:385] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.641158 11553 raft_consensus.cc:740] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:36.645047 10588 master_service.cc:960] ReplaceTablet: received request to replace tablet 135d3d65ea7846d3bc2cb59d44dbedf6 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:36.641950 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 6d35f291c92a45899eed0f2f10a3a1d8 (DEFAULT_TABLE table=default.Table_14613c38aa0e4b258598254918b193bb [id=fa2e19a3042b41959165ca54e46ca690]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.646660 11949 ts_tablet_manager.cc:1929] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:36.647147 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d35f291c92a45899eed0f2f10a3a1d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.647541 11553 consensus_queue.cc:260] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:36.653672 11949 log.cc:1199] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/98bee541fc1345fa83ffebeba05d8c8e
I20260809 01:45:36.654703 11949 ts_tablet_manager.cc:1950] T 98bee541fc1345fa83ffebeba05d8c8e P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:36.657701 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet 98bee541fc1345fa83ffebeba05d8c8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:35 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:36.658548 11553 ts_tablet_manager.cc:1434] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.021s user 0.004s sys 0.004s
I20260809 01:45:36.659385 11553 tablet_bootstrap.cc:492] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:36.663386 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 98bee541fc1345fa83ffebeba05d8c8e (table default.Table_e212a7cfad494f2d97ac5eada6ac15f5 [id=d2c591183b4646ca88b53698200a62c2]) successfully deleted
I20260809 01:45:36.663848 11951 ts_tablet_manager.cc:1916] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:36.669216 11553 tablet_bootstrap.cc:654] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:36.670446 11063 consensus_peers.cc:597] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 61c83b8bcf54461bb24565e3a989100a. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:36.671720 11624 raft_consensus.cc:493] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:36.672364 11624 raft_consensus.cc:515] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.675401 11624 leader_election.cc:290] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:36.677639 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da86b5e4a27544c59ae6bb387440d246" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:36.683790 11910 raft_consensus.cc:515] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:36.686091 10805 leader_election.cc:343] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.687768 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet 98bee541fc1345fa83ffebeba05d8c8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:35 UTC) from {username='slave'} at 127.0.0.1:45982
W20260809 01:45:36.689252 10591 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 25) took 1449 ms (client timeout 1000 ms). Trace:
W20260809 01:45:36.690686 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 98bee541fc1345fa83ffebeba05d8c8e because the tablet was not found. No further retry: Not found: Tablet not found: 98bee541fc1345fa83ffebeba05d8c8e
I20260809 01:45:36.691313 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet 34b103e2d3854eb4bf8e5cd649cbe129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:31 UTC) from {username='slave'} at 127.0.0.1:45982
W20260809 01:45:36.689599 10591 rpcz_store.cc:269] 0809 01:45:35.239946 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:35.241591 (+ 1645us) service_pool.cc:224] Handling call
0809 01:45:36.689184 (+1447593us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":10}
I20260809 01:45:36.693414 11949 tablet_replica.cc:333] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:36.694808 11949 raft_consensus.cc:2243] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:36.695940 11949 pending_rounds.cc:70] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8: Trying to abort 1 pending ops.
I20260809 01:45:36.626806 11950 ts_tablet_manager.cc:1916] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:36.696348 11949 pending_rounds.cc:77] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438777462919168 op_type: NO_OP noop_request { }
I20260809 01:45:36.697831 11949 raft_consensus.cc:2889] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:36.677496 10586 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233]). Expected version 2 got 0
I20260809 01:45:36.698307 11949 raft_consensus.cc:2272] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:36.707769 11949 ts_tablet_manager.cc:1916] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:36.628759 11970 raft_consensus.cc:515] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.727993 11553 tablet_bootstrap.cc:492] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:36.728477 11553 ts_tablet_manager.cc:1403] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.069s user 0.004s sys 0.010s
I20260809 01:45:36.700809 11554 tablet_bootstrap.cc:654] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:36.702934 11354 tablet_service.cc:1511] Processing CreateTablet for tablet c16897857df24c6da5744a99d5f687c0 (DEFAULT_TABLE table=default.Table_6d4e0b3afc864450bdf1a8fb35f7dc75 [id=9b307bc98a124b07a2f46fafb20ce1f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.730500 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16897857df24c6da5744a99d5f687c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.704079 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet 43d84f310d8e4df7b8b2d602289025ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:36.705358 11364 tablet_service.cc:1511] Processing CreateTablet for tablet c2f86bba4bd94fbcacdc9c139e9de2d7 (DEFAULT_TABLE table=default.Table_5f4d617a86c2487fa9f64cb58f14de3c [id=f64daa5a2706436081e9166a5f3a0cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.707774 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 7706838afc9f4dbebc3213b82c4da126 (DEFAULT_TABLE table=default.Table_57ef64be310d4250b77b141860e944d5 [id=212edfae1bf849c3b46674ec191ee76d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.731055 11553 raft_consensus.cc:359] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:36.731768 11553 raft_consensus.cc:385] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.732052 11553 raft_consensus.cc:740] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:36.735357 10871 tablet_service.cc:1511] Processing CreateTablet for tablet b6159d660d544b3e907a08372027c0e2 (DEFAULT_TABLE table=default.Table_59150b4860b64269a5f1712f8b8f0d30 [id=ddfe680283854b9484dc3e9d6e626ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.736330 10873 tablet_service.cc:1511] Processing CreateTablet for tablet 60d356fcc48441ca80f2df29b533404f (DEFAULT_TABLE table=default.Table_29a4bd6bd10a473dbd9dbda008b9eb16 [id=67ee87fabc7749279341528fbcce16d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.737016 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6159d660d544b3e907a08372027c0e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.739627 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60d356fcc48441ca80f2df29b533404f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.748076 11624 raft_consensus.cc:493] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:36.748688 11624 raft_consensus.cc:515] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.752545 11624 leader_election.cc:290] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:36.745849 11553 consensus_queue.cc:260] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:36.755410 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac6eded297214b58a5d8129a60ca1120" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:36.755967 11553 ts_tablet_manager.cc:1434] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.027s user 0.006s sys 0.002s
I20260809 01:45:36.762830 11553 tablet_bootstrap.cc:492] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:36.801139 11553 tablet_bootstrap.cc:654] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:36.804672 11157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac6eded297214b58a5d8129a60ca1120" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:36.806728 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da86b5e4a27544c59ae6bb387440d246" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:36.807958 11949 ts_tablet_manager.cc:1929] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:36.808595 11949 log.cc:1199] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/34b103e2d3854eb4bf8e5cd649cbe129
I20260809 01:45:36.809536 11949 ts_tablet_manager.cc:1950] T 34b103e2d3854eb4bf8e5cd649cbe129 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
W20260809 01:45:36.810140 10804 leader_election.cc:343] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:36.812594 10804 leader_election.cc:343] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.812981 10804 leader_election.cc:304] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:36.814216 11624 raft_consensus.cc:2749] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:36.794035 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb5e2ec60c24f8396daed992ca314ac" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:36.828745 11553 tablet_bootstrap.cc:492] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:36.829141 11553 ts_tablet_manager.cc:1403] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.067s user 0.010s sys 0.004s
I20260809 01:45:36.754240 11364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f86bba4bd94fbcacdc9c139e9de2d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.831566 11553 raft_consensus.cc:359] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:36.832167 11553 raft_consensus.cc:385] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.832401 11553 raft_consensus.cc:740] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:36.832414 11119 tablet_service.cc:1558] Processing DeleteTablet for tablet 43d84f310d8e4df7b8b2d602289025ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:36.835366 11553 consensus_queue.cc:260] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:36.835413 11126 tablet_service.cc:1511] Processing CreateTablet for tablet b6159d660d544b3e907a08372027c0e2 (DEFAULT_TABLE table=default.Table_59150b4860b64269a5f1712f8b8f0d30 [id=ddfe680283854b9484dc3e9d6e626ab9]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:36.768955 11289 leader_election.cc:343] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 5bb5e2ec60c24f8396daed992ca314ac
W20260809 01:45:36.838078 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 69167.7us
I20260809 01:45:36.837348 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6159d660d544b3e907a08372027c0e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.847656 11553 ts_tablet_manager.cc:1434] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.018s user 0.006s sys 0.004s
I20260809 01:45:36.849401 11553 tablet_bootstrap.cc:492] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:36.850124 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0da8d25290814e66b99c2cc1d5f8b0cf" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:36.851042 10905 raft_consensus.cc:3060] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:36.791760 11970 leader_election.cc:290] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:36.797510 11352 tablet_service.cc:1511] Processing CreateTablet for tablet b6159d660d544b3e907a08372027c0e2 (DEFAULT_TABLE table=default.Table_59150b4860b64269a5f1712f8b8f0d30 [id=ddfe680283854b9484dc3e9d6e626ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:36.858067 11553 tablet_bootstrap.cc:654] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:36.858932 11352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6159d660d544b3e907a08372027c0e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.859412 11888 consensus_queue.cc:1048] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 01:45:36.868624 11120 consensus_meta.cc:287] T cef88ffe9e95437da81793795742def3 P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 0.507s user 0.008s sys 0.004s
I20260809 01:45:36.874189 11553 tablet_bootstrap.cc:492] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:36.874584 11553 ts_tablet_manager.cc:1403] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.025s user 0.013s sys 0.000s
I20260809 01:45:36.877744 11553 raft_consensus.cc:359] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:36.878376 11553 raft_consensus.cc:385] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.878605 11553 raft_consensus.cc:740] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:36.879247 11553 consensus_queue.cc:260] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:36.881143 11553 ts_tablet_manager.cc:1434] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260809 01:45:36.883574 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7706838afc9f4dbebc3213b82c4da126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:36.888211 11553 tablet_bootstrap.cc:492] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
W20260809 01:45:36.890717 10805 leader_election.cc:343] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.891528 10805 leader_election.cc:304] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:36.892629 11624 raft_consensus.cc:2749] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:36.903054 11551 tablet_bootstrap.cc:492] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:36.903548 11551 ts_tablet_manager.cc:1403] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.946s user 0.008s sys 0.012s
I20260809 01:45:36.904742 11553 tablet_bootstrap.cc:654] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:36.912710 11624 raft_consensus.cc:493] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:36.815537 11910 leader_election.cc:290] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:36.913444 11624 raft_consensus.cc:515] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.918730 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "201329c676ec4b6abbb72b22519bc41d" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:36.920166 10804 leader_election.cc:343] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.921411 11624 leader_election.cc:290] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:36.922071 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 34b103e2d3854eb4bf8e5cd649cbe129 (table default.Table_175d0b46158d49a88aac3833ba31533a [id=cf3f859364164b9085d525891266a825]) successfully deleted
I20260809 01:45:36.923020 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "201329c676ec4b6abbb72b22519bc41d" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:36.924531 10805 leader_election.cc:343] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.924955 10805 leader_election.cc:304] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:36.925668 11916 raft_consensus.cc:493] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:36.926054 11916 raft_consensus.cc:515] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:36.928622 10596 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.66:57751 (request call id 23) took 1168 ms (client timeout 1000 ms). Trace:
I20260809 01:45:36.928936 11553 tablet_bootstrap.cc:492] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:36.929312 11553 ts_tablet_manager.cc:1403] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.041s user 0.004s sys 0.012s
W20260809 01:45:36.929144 10596 rpcz_store.cc:269] 0809 01:45:35.760075 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:35.827873 (+ 67798us) service_pool.cc:224] Handling call
0809 01:45:36.928525 (+1100652us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":150729,"reported_tablets":9}
I20260809 01:45:36.932528 11553 raft_consensus.cc:359] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:36.933094 11553 raft_consensus.cc:385] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.933339 11553 raft_consensus.cc:740] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:36.933934 11553 consensus_queue.cc:260] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:36.934954 11960 raft_consensus.cc:2749] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:36.935315 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135d3d65ea7846d3bc2cb59d44dbedf6" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:36.937182 11553 ts_tablet_manager.cc:1434] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
W20260809 01:45:36.937512 10804 leader_election.cc:343] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.937986 11553 tablet_bootstrap.cc:492] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:36.938866 11916 leader_election.cc:290] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:36.945283 11553 tablet_bootstrap.cc:654] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:36.947158 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135d3d65ea7846d3bc2cb59d44dbedf6" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:36.951205 10805 leader_election.cc:343] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.951730 10805 leader_election.cc:304] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:36.952800 11916 raft_consensus.cc:2749] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:36.956053 11553 tablet_bootstrap.cc:492] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:36.956416 11553 ts_tablet_manager.cc:1403] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.009s
I20260809 01:45:36.958729 11553 raft_consensus.cc:359] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:36.959539 11553 raft_consensus.cc:385] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.959802 11553 raft_consensus.cc:740] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:36.959839 11916 raft_consensus.cc:493] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:36.960381 11553 consensus_queue.cc:260] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:36.960404 11916 raft_consensus.cc:515] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.963696 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737d7047a0c74a03b3203112987c0cb2" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:36.963959 11553 ts_tablet_manager.cc:1434] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260809 01:45:36.965287 11553 tablet_bootstrap.cc:492] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
W20260809 01:45:36.966038 10804 leader_election.cc:343] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.966928 11916 leader_election.cc:290] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:36.969230 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737d7047a0c74a03b3203112987c0cb2" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:36.970726 10805 leader_election.cc:343] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.971330 10805 leader_election.cc:304] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:36.971138 11553 tablet_bootstrap.cc:654] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:36.972453 11283 leader_election.cc:343] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.972545 11916 raft_consensus.cc:2749] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:36.973093 11283 leader_election.cc:304] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:36.974210 11910 raft_consensus.cc:2749] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:36.976804 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0da8d25290814e66b99c2cc1d5f8b0cf" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
W20260809 01:45:36.978086 11283 leader_election.cc:343] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:36.978381 11910 raft_consensus.cc:493] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:36.978809 11910 raft_consensus.cc:515] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.981349 11910 leader_election.cc:290] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:36.983223 11553 tablet_bootstrap.cc:492] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:36.983613 11553 ts_tablet_manager.cc:1403] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.005s
I20260809 01:45:36.984050 11551 raft_consensus.cc:359] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.984786 11551 raft_consensus.cc:385] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.985045 11551 raft_consensus.cc:740] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:36.985824 11553 raft_consensus.cc:359] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:36.985896 11551 consensus_queue.cc:260] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:36.986404 11553 raft_consensus.cc:385] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:36.986639 11553 raft_consensus.cc:740] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:36.987210 11553 consensus_queue.cc:260] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:36.988871 11553 ts_tablet_manager.cc:1434] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 01:45:36.990460 11553 tablet_bootstrap.cc:492] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:36.992368 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aac67e7819f4a70b1050b558868f27f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:36.992987 10910 raft_consensus.cc:2468] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:36.994161 11289 leader_election.cc:304] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:36.995278 11910 raft_consensus.cc:2804] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:36.995659 11910 raft_consensus.cc:493] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:36.996040 11910 raft_consensus.cc:3060] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:36.996968 11127 consensus_meta.cc:287] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 0.659s user 0.009s sys 0.000s
I20260809 01:45:37.000981 11553 tablet_bootstrap.cc:654] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:37.013154 11916 raft_consensus.cc:493] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:37.013723 11916 raft_consensus.cc:515] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:37.014981 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aac67e7819f4a70b1050b558868f27f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:37.015352 11553 tablet_bootstrap.cc:492] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:37.015748 11553 ts_tablet_manager.cc:1403] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.025s user 0.015s sys 0.003s
I20260809 01:45:37.015885 11916 leader_election.cc:290] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:37.017442 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389407e7ff464e0d8e84bbdbc4c6081e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:37.018033 11553 raft_consensus.cc:359] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:37.018678 11553 raft_consensus.cc:385] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:37.018929 11553 raft_consensus.cc:740] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
W20260809 01:45:37.019340 10804 leader_election.cc:343] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:37.019537 11553 consensus_queue.cc:260] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:37.020218 11916 raft_consensus.cc:493] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:37.020653 11916 raft_consensus.cc:515] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:37.021431 11553 ts_tablet_manager.cc:1434] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 01:45:37.022258 11553 tablet_bootstrap.cc:492] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:37.023453 10905 raft_consensus.cc:2468] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:37.023592 11916 leader_election.cc:290] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:37.024269 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0d79e03e66410487a72f1b47b327a0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:37.025581 10804 leader_election.cc:343] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:37.027318 11553 tablet_bootstrap.cc:654] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:37.032893 10597 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:37956 (request call id 129) took 1216 ms. Trace:
I20260809 01:45:37.034873 10589 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:37956 (request call id 131) took 1100 ms. Trace:
I20260809 01:45:37.033257 10597 rpcz_store.cc:276] 0809 01:45:35.816588 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.235095 (+418507us) service_pool.cc:224] Handling call
0809 01:45:36.237453 (+ 2358us) catalog_manager.cc:2127] Acquired catalog manager lock
0809 01:45:36.419292 (+181839us) catalog_manager.cc:2173] Created new table and tablet info
0809 01:45:37.032218 (+612926us) catalog_manager.cc:2233] Wrote table and tablets to system table
0809 01:45:37.032540 (+ 322us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0809 01:45:37.032830 (+ 290us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:36.423491 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:36.423844 (+ 353us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:36.423877 (+ 33us) write_op.cc:435] Acquired schema lock
0809 01:45:36.423894 (+ 17us) tablet.cc:662] Decoding operations
0809 01:45:36.424152 (+ 258us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:36.424187 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:36.424211 (+ 24us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:36.424392 (+ 181us) tablet.cc:701] Row locks acquired
0809 01:45:36.424404 (+ 12us) write_op.cc:260] PREPARE: finished
0809 01:45:36.424497 (+ 93us) write_op.cc:270] Start()
0809 01:45:36.424571 (+ 74us) write_op.cc:276] Timestamp: P: 1786239936414282 usec, L: 0
0809 01:45:36.424586 (+ 15us) op_driver.cc:348] REPLICATION: starting
0809 01:45:36.425040 (+ 454us) log.cc:844] Serialized 433 byte log entry
0809 01:45:37.028520 (+603480us) op_driver.cc:464] REPLICATION: finished
0809 01:45:37.030882 (+ 2362us) write_op.cc:301] APPLY: starting
0809 01:45:37.030928 (+ 46us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:37.031024 (+ 96us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:37.031116 (+ 92us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:37.031485 (+ 369us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:37.031610 (+ 125us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 01:45:37.031652 (+ 42us) write_op.cc:312] APPLY: finished
0809 01:45:37.031822 (+ 170us) log.cc:844] Serialized 29 byte log entry
0809 01:45:37.031948 (+ 126us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:37.032005 (+ 57us) write_op.cc:454] Released schema lock
0809 01:45:37.032032 (+ 27us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2248,"apply.run_cpu_time_us":1327,"apply.run_wall_time_us":1327,"num_ops":2,"prepare.queue_time_us":3087,"prepare.run_cpu_time_us":1732,"prepare.run_wall_time_us":2639,"replication_time_us":603860,"threads_started":1}]]}
I20260809 01:45:37.040251 11916 raft_consensus.cc:493] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:37.040716 11916 raft_consensus.cc:515] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:37.041630 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389407e7ff464e0d8e84bbdbc4c6081e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:37.045122 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4754f2e2a35f4a6a91fa333e2cf564e4" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:37.046542 10804 leader_election.cc:343] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:37.049074 11916 leader_election.cc:290] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:37.053332 11283 leader_election.cc:343] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:37.050688 11391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0d79e03e66410487a72f1b47b327a0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:37.062947 11554 tablet_bootstrap.cc:492] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:37.063421 11554 ts_tablet_manager.cc:1403] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.612s user 0.011s sys 0.004s
W20260809 01:45:37.063791 10805 leader_election.cc:343] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:37.064666 10805 leader_election.cc:304] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:37.065981 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4754f2e2a35f4a6a91fa333e2cf564e4" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:37.079638 11551 ts_tablet_manager.cc:1434] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.176s user 0.006s sys 0.000s
I20260809 01:45:37.035190 10589 rpcz_store.cc:276] 0809 01:45:35.934079 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.367309 (+433230us) service_pool.cc:224] Handling call
0809 01:45:36.399814 (+ 32505us) catalog_manager.cc:2127] Acquired catalog manager lock
0809 01:45:36.413326 (+ 13512us) catalog_manager.cc:2173] Created new table and tablet info
0809 01:45:37.034218 (+620892us) catalog_manager.cc:2233] Wrote table and tablets to system table
0809 01:45:37.034540 (+ 322us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0809 01:45:37.034815 (+ 275us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:36.414692 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:36.415143 (+ 451us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:36.415165 (+ 22us) write_op.cc:435] Acquired schema lock
0809 01:45:36.415178 (+ 13us) tablet.cc:662] Decoding operations
0809 01:45:36.416199 (+ 1021us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:36.416236 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:36.416258 (+ 22us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:36.416460 (+ 202us) tablet.cc:701] Row locks acquired
0809 01:45:36.416473 (+ 13us) write_op.cc:260] PREPARE: finished
0809 01:45:36.416572 (+ 99us) write_op.cc:270] Start()
0809 01:45:36.416657 (+ 85us) write_op.cc:276] Timestamp: P: 1786239936406357 usec, L: 0
0809 01:45:36.416673 (+ 16us) op_driver.cc:348] REPLICATION: starting
0809 01:45:36.417154 (+ 481us) log.cc:844] Serialized 433 byte log entry
0809 01:45:37.028263 (+611109us) op_driver.cc:464] REPLICATION: finished
0809 01:45:37.028961 (+ 698us) write_op.cc:301] APPLY: starting
0809 01:45:37.029031 (+ 70us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:37.029136 (+ 105us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:37.029258 (+ 122us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:37.029686 (+ 428us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:37.029820 (+ 134us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 01:45:37.029867 (+ 47us) write_op.cc:312] APPLY: finished
0809 01:45:37.030054 (+ 187us) log.cc:844] Serialized 29 byte log entry
0809 01:45:37.030271 (+ 217us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:37.030337 (+ 66us) write_op.cc:454] Released schema lock
0809 01:45:37.030373 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:37.030602 (+ 229us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:37.030635 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":524,"apply.run_cpu_time_us":1621,"apply.run_wall_time_us":1622,"num_ops":2,"prepare.queue_time_us":102,"prepare.run_cpu_time_us":1965,"prepare.run_wall_time_us":8680,"replication_time_us":611471}]]}
I20260809 01:45:37.130975 11554 raft_consensus.cc:359] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:37.131676 11554 raft_consensus.cc:385] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:37.131950 11554 raft_consensus.cc:740] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:37.148236 11551 tablet_bootstrap.cc:492] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:37.154384 11624 raft_consensus.cc:2749] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:37.156309 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 92565.8us
W20260809 01:45:37.105029 11123 consensus_meta.cc:287] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 0.552s user 0.004s sys 0.004s
W20260809 01:45:37.157433 10805 leader_election.cc:343] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:37.158006 10805 leader_election.cc:304] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:37.164568 11910 raft_consensus.cc:515] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:37.166034 10579 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 36a542c773b7438ead16c5b3af224cf9 (table default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 36a542c773b7438ead16c5b3af224cf9 has no consensus state
I20260809 01:45:37.181809 10597 master_service.cc:960] ReplaceTablet: received request to replace tablet 34b103e2d3854eb4bf8e5cd649cbe129 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:37.222771 10599 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:37956 (request call id 132) took 1184 ms. Trace:
I20260809 01:45:37.226706 10598 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet af4b3d2f3aa540879c47eaa541df8b65
I20260809 01:45:37.227222 11624 raft_consensus.cc:2749] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:37.223111 10599 rpcz_store.cc:276] 0809 01:45:36.038352 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.393376 (+355024us) service_pool.cc:224] Handling call
0809 01:45:36.441823 (+ 48447us) catalog_manager.cc:2127] Acquired catalog manager lock
0809 01:45:36.499534 (+ 57711us) catalog_manager.cc:2173] Created new table and tablet info
0809 01:45:37.222113 (+722579us) catalog_manager.cc:2233] Wrote table and tablets to system table
0809 01:45:37.222442 (+ 329us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0809 01:45:37.222710 (+ 268us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:36.500999 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:36.501378 (+ 379us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:36.501402 (+ 24us) write_op.cc:435] Acquired schema lock
0809 01:45:36.501415 (+ 13us) tablet.cc:662] Decoding operations
0809 01:45:36.501750 (+ 335us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:36.501798 (+ 48us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:36.501822 (+ 24us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:36.502046 (+ 224us) tablet.cc:701] Row locks acquired
0809 01:45:36.502058 (+ 12us) write_op.cc:260] PREPARE: finished
0809 01:45:36.502169 (+ 111us) write_op.cc:270] Start()
0809 01:45:36.502273 (+ 104us) write_op.cc:276] Timestamp: P: 1786239936491943 usec, L: 0
0809 01:45:36.502290 (+ 17us) op_driver.cc:348] REPLICATION: starting
0809 01:45:36.502894 (+ 604us) log.cc:844] Serialized 433 byte log entry
0809 01:45:37.216508 (+713614us) op_driver.cc:464] REPLICATION: finished
0809 01:45:37.219185 (+ 2677us) write_op.cc:301] APPLY: starting
0809 01:45:37.219264 (+ 79us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:37.219363 (+ 99us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:37.219459 (+ 96us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:37.219849 (+ 390us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:37.220023 (+ 174us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 01:45:37.220058 (+ 35us) write_op.cc:312] APPLY: finished
0809 01:45:37.220281 (+ 223us) log.cc:844] Serialized 29 byte log entry
0809 01:45:37.220443 (+ 162us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:37.220509 (+ 66us) write_op.cc:454] Released schema lock
0809 01:45:37.220552 (+ 43us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:37.220759 (+ 207us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:37.220787 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2495,"apply.run_cpu_time_us":1558,"apply.run_wall_time_us":1558,"num_ops":2,"prepare.queue_time_us":120,"prepare.run_cpu_time_us":2101,"prepare.run_wall_time_us":3133,"replication_time_us":714121}]]}
W20260809 01:45:37.230391 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 73113.6us
W20260809 01:45:37.231189 10805 leader_election.cc:343] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:37.231602 10805 leader_election.cc:304] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:37.233369 10592 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e3684fa56f9549f492a96a5c765eda78 has no consensus state
I20260809 01:45:37.238895 10588 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 135d3d65ea7846d3bc2cb59d44dbedf6
I20260809 01:45:37.240499 10588 catalog_manager.cc:6807] ReplaceTablet: tablet 135d3d65ea7846d3bc2cb59d44dbedf6 deleted and replaced by tablet c528b45de0d14c19af5d6836b7a14f1c
I20260809 01:45:37.244942 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 135d3d65ea7846d3bc2cb59d44dbedf6 with delete_type TABLET_DATA_DELETED (replaced by tablet c528b45de0d14c19af5d6836b7a14f1c) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:37.245913 11949 tablet_replica.cc:333] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:37.246805 11949 raft_consensus.cc:2243] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260809 01:45:37.253322 10595 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cfb9afd3be194830ba99f5e20fdaca0a has no consensus state
W20260809 01:45:37.132974 11063 consensus_peers.cc:597] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43d84f310d8e4df7b8b2d602289025ae. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:37.246716 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet 135d3d65ea7846d3bc2cb59d44dbedf6 with delete_type TABLET_DATA_DELETED (replaced by tablet c528b45de0d14c19af5d6836b7a14f1c) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:37.248157 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 60d356fcc48441ca80f2df29b533404f (DEFAULT_TABLE table=default.Table_29a4bd6bd10a473dbd9dbda008b9eb16 [id=67ee87fabc7749279341528fbcce16d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:37.256970 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60d356fcc48441ca80f2df29b533404f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:37.259764 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_2b01d8bf86d44f96b64752f511b02f50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:37.262708 10589 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_fe1acd943c5c42c9b09a0a5845cbee68" } modify_external_catalogs: true
I20260809 01:45:37.263402 10589 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_fe1acd943c5c42c9b09a0a5845cbee68" } modify_external_catalogs: true
W20260809 01:45:37.261381 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b01d8bf86d44f96b64752f511b02f50 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:37.192509 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 60d356fcc48441ca80f2df29b533404f (DEFAULT_TABLE table=default.Table_29a4bd6bd10a473dbd9dbda008b9eb16 [id=67ee87fabc7749279341528fbcce16d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:37.241238 11354 tablet_service.cc:1558] Processing DeleteTablet for tablet 135d3d65ea7846d3bc2cb59d44dbedf6 with delete_type TABLET_DATA_DELETED (replaced by tablet c528b45de0d14c19af5d6836b7a14f1c) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:37.291011 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60d356fcc48441ca80f2df29b533404f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:37.269618 11063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 136772us
I20260809 01:45:37.300985 11949 raft_consensus.cc:2272] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:37.302042 11624 raft_consensus.cc:2749] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:37.304733 11949 ts_tablet_manager.cc:1916] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:37.258491 11554 consensus_queue.cc:260] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:37.321748 11554 ts_tablet_manager.cc:1434] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.258s user 0.009s sys 0.000s
I20260809 01:45:37.322695 11554 tablet_bootstrap.cc:492] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:37.333666 11624 raft_consensus.cc:493] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:37.334496 11624 raft_consensus.cc:515] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:37.338639 10591 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 5f8824953c794450ba274b160d891fcb (table default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]). Expected version 1 got 0
I20260809 01:45:37.338719 11624 leader_election.cc:290] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:37.339793 10591 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233]). Expected version 2 got 0
W20260809 01:45:37.341996 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 110838us
I20260809 01:45:37.343655 11289 leader_election.cc:304] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:37.344731 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aac67e7819f4a70b1050b558868f27f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:37.345186 11910 leader_election.cc:290] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:37.345608 10905 raft_consensus.cc:3060] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:37.345916 11910 raft_consensus.cc:2804] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:37.346383 11910 raft_consensus.cc:697] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:37.347369 11910 consensus_queue.cc:237] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:37.349373 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef8a334741d3419eaa377338e0eaa889" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:37.352839 11951 ts_tablet_manager.cc:1929] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:37.354270 11624 raft_consensus.cc:493] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:37.355152 11553 tablet_bootstrap.cc:492] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:37.355598 11553 ts_tablet_manager.cc:1403] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.334s user 0.076s sys 0.135s
I20260809 01:45:37.355027 11624 raft_consensus.cc:515] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:37.358546 11553 raft_consensus.cc:359] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:37.359122 11553 raft_consensus.cc:385] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:37.359354 11553 raft_consensus.cc:740] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:37.359493 11624 leader_election.cc:290] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:37.359911 11553 consensus_queue.cc:260] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:37.367466 11553 ts_tablet_manager.cc:1434] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20260809 01:45:37.368301 11553 tablet_bootstrap.cc:492] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
W20260809 01:45:37.377102 10576 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 106218us
W20260809 01:45:37.377800 10579 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 90512.1us
W20260809 01:45:37.379760 10586 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.67:40463 (request call id 16) took 1216 ms (client timeout 1000 ms). Trace:
W20260809 01:45:37.380123 10586 rpcz_store.cc:269] 0809 01:45:36.163168 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.173292 (+ 10124us) service_pool.cc:224] Handling call
0809 01:45:37.379645 (+1206353us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63346,"reported_tablets":12}
I20260809 01:45:37.380976 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_f84f34b0360549f39d13b657f425307c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:37.382516 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f84f34b0360549f39d13b657f425307c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:37.383529 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_15f337a56a7d4b7aa19d18c0f036799c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:37.385456 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15f337a56a7d4b7aa19d18c0f036799c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:37.396162 11364 tablet_service.cc:1558] Processing DeleteTablet for tablet 34b103e2d3854eb4bf8e5cd649cbe129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:31 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:37.397753 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 5bb5e2ec60c24f8396daed992ca314ac with delete_type TABLET_DATA_DELETED (replaced by tablet c2f86bba4bd94fbcacdc9c139e9de2d7) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:37.403338 11976 consensus_queue.cc:1048] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:37.409955 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637f0d6261f041b5aeff5fcc709a3dfc" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:37.411260 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 5050a590fe0240ac8c9f9191ec2475a1
I20260809 01:45:37.407228 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aac67e7819f4a70b1050b558868f27f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:37.409389 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef8a334741d3419eaa377338e0eaa889" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:37.415648 11283 leader_election.cc:343] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:37.418756 10805 leader_election.cc:343] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:37.353412 11951 log.cc:1199] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/48b7459ad0974cb9948d647459b22260
W20260809 01:45:37.425984 10804 leader_election.cc:343] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:37.426982 10804 leader_election.cc:343] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:37.427066 11951 ts_tablet_manager.cc:1950] T 48b7459ad0974cb9948d647459b22260 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:37.427397 10804 leader_election.cc:304] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:37.428694 11960 raft_consensus.cc:2749] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:37.430893 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet 5050a590fe0240ac8c9f9191ec2475a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:45982
W20260809 01:45:37.431857 11063 consensus_peers.cc:597] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 98bee541fc1345fa83ffebeba05d8c8e. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:37.435707 11976 consensus_queue.cc:1048] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:37.437088 11125 tablet_service.cc:1558] Processing DeleteTablet for tablet 5050a590fe0240ac8c9f9191ec2475a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:56566
W20260809 01:45:37.321702 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 77641.7us
I20260809 01:45:37.440155 10598 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_6df59952b936458893928a28040decda" } new_table_name: "default.Table_a141ed06432c4f2bad256b8a687a39e4" modify_external_catalogs: true
I20260809 01:45:37.457111 10588 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_a0d298080721403d9305bc761dda44a9" } new_table_name: "default.Table_334efa6e437847f9b745246d8a442b89" modify_external_catalogs: true
I20260809 01:45:37.464879 12007 consensus_queue.cc:1048] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:37.466434 12007 consensus_queue.cc:1048] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:37.465143 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_63e5e407241d4eaea93f27d8285ab33a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:37.469408 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 5050a590fe0240ac8c9f9191ec2475a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:53476
W20260809 01:45:37.470439 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_63e5e407241d4eaea93f27d8285ab33a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:37.473814 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637f0d6261f041b5aeff5fcc709a3dfc" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:37.475548 10805 leader_election.cc:343] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:37.476250 10805 leader_election.cc:304] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:37.477190 11916 raft_consensus.cc:2749] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:37.490427 10905 raft_consensus.cc:2468] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:37.493031 11554 tablet_bootstrap.cc:654] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:37.500890 11553 tablet_bootstrap.cc:654] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:37.504005 11289 leader_election.cc:304] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:37.505849 11976 raft_consensus.cc:2804] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:37.506604 11976 raft_consensus.cc:697] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:37.507804 11976 consensus_queue.cc:237] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:37.510250 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 83) took 1415 ms. Trace:
I20260809 01:45:37.511058 11951 rpcz_store.cc:276] 0809 01:45:36.094533 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.094725 (+ 192us) service_pool.cc:224] Handling call
0809 01:45:36.403494 (+308769us) ts_tablet_manager.cc:1151] Deleting tablet 48b7459ad0974cb9948d647459b22260
0809 01:45:36.403539 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:37.095833 (+692294us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:37.352718 (+256885us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:37.510182 (+157464us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84937,"tablet-delete.queue_time_us":307829}
I20260809 01:45:37.513545 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 48b7459ad0974cb9948d647459b22260 (table default.Table_d37c43f0673044eb826f3944101bcd4b [id=ec2be55379204f57a7bf8a8739b73033]) successfully deleted
I20260809 01:45:37.526067 11125 tablet_service.cc:1558] Processing DeleteTablet for tablet 34b103e2d3854eb4bf8e5cd649cbe129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:31 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:37.553391 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 84) took 1458 ms. Trace:
I20260809 01:45:37.556592 11951 rpcz_store.cc:276] 0809 01:45:36.095077 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.095247 (+ 170us) service_pool.cc:224] Handling call
0809 01:45:37.513291 (+1418044us) ts_tablet_manager.cc:1151] Deleting tablet e155d16b1d594a789f539390a44bc39f
0809 01:45:37.513425 (+ 134us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:37.553312 (+ 39887us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1417212}
I20260809 01:45:37.559957 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 85) took 1463 ms. Trace:
W20260809 01:45:37.563069 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet e155d16b1d594a789f539390a44bc39f because the tablet was not found. No further retry: Not found: Tablet not found: e155d16b1d594a789f539390a44bc39f
I20260809 01:45:37.570303 11624 raft_consensus.cc:493] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:37.570622 10588 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ac6eded297214b58a5d8129a60ca1120 consensus state has no leader
I20260809 01:45:37.571009 11624 raft_consensus.cc:515] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:37.575096 11624 leader_election.cc:290] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:37.576802 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d1ed215ba644abbae09e78016b2c21" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:37.578841 10805 leader_election.cc:343] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:37.603000 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d1ed215ba644abbae09e78016b2c21" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:37.604935 10804 leader_election.cc:343] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:37.605429 10804 leader_election.cc:304] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:37.607165 11624 raft_consensus.cc:2749] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:37.564591 11951 rpcz_store.cc:276] 0809 01:45:36.096564 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.100483 (+ 3919us) service_pool.cc:224] Handling call
0809 01:45:37.559347 (+1458864us) ts_tablet_manager.cc:1151] Deleting tablet 2674c9dbf9e44d63b41427f2fda4e311
0809 01:45:37.559429 (+ 82us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:37.559874 (+ 445us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1457902}
I20260809 01:45:37.622404 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_e03575a4e6bb46afb0346fad51b99eb5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:37.626202 10597 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_8a405d19ab94416c8d58fff878edae0a" } modify_external_catalogs: true
I20260809 01:45:37.626909 10597 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_8a405d19ab94416c8d58fff878edae0a" } modify_external_catalogs: true
W20260809 01:45:37.627316 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 2674c9dbf9e44d63b41427f2fda4e311 because the tablet was not found. No further retry: Not found: Tablet not found: 2674c9dbf9e44d63b41427f2fda4e311
W20260809 01:45:37.628810 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e03575a4e6bb46afb0346fad51b99eb5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:37.632432 11553 tablet_bootstrap.cc:492] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:37.632584 12007 raft_consensus.cc:493] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:37.632944 11553 ts_tablet_manager.cc:1403] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.265s user 0.007s sys 0.007s
I20260809 01:45:37.633392 12007 raft_consensus.cc:515] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:37.636691 12007 leader_election.cc:290] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:37.639884 11063 leader_election.cc:343] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 34b103e2d3854eb4bf8e5cd649cbe129
I20260809 01:45:37.645203 11551 tablet_bootstrap.cc:654] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:37.645707 11743 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:45:37.650051 11743 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260809 01:45:37.651399 11553 raft_consensus.cc:359] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:37.652058 11553 raft_consensus.cc:385] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:37.652316 11553 raft_consensus.cc:740] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
W20260809 01:45:37.653060 11743 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 01:45:37.635800 11950 ts_tablet_manager.cc:1929] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:37.654191 11950 log.cc:1199] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/48b7459ad0974cb9948d647459b22260
I20260809 01:45:37.655403 11950 ts_tablet_manager.cc:1950] T 48b7459ad0974cb9948d647459b22260 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:37.658941 11950 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 75) took 1581 ms. Trace:
I20260809 01:45:37.659533 11950 rpcz_store.cc:276] 0809 01:45:36.078070 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.078312 (+ 242us) service_pool.cc:224] Handling call
0809 01:45:36.401232 (+322920us) ts_tablet_manager.cc:1151] Deleting tablet 48b7459ad0974cb9948d647459b22260
0809 01:45:36.401329 (+ 97us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:37.107408 (+706079us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:37.635678 (+528270us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:37.658853 (+ 23175us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":321869}
I20260809 01:45:37.660401 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b103e2d3854eb4bf8e5cd649cbe129" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:37.661316 11395 raft_consensus.cc:2393] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33b5652538e64ed58a09993d3188f8d7 in current term 1: Already voted for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in this term.
I20260809 01:45:37.661698 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 48b7459ad0974cb9948d647459b22260 (table default.Table_d37c43f0673044eb826f3944101bcd4b [id=ec2be55379204f57a7bf8a8739b73033]) successfully deleted
I20260809 01:45:37.662899 11950 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 78) took 1492 ms. Trace:
I20260809 01:45:37.643955 11951 tablet_replica.cc:333] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:37.663417 11950 rpcz_store.cc:276] 0809 01:45:36.170614 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.171156 (+ 542us) service_pool.cc:224] Handling call
0809 01:45:37.662578 (+1491422us) ts_tablet_manager.cc:1151] Deleting tablet e155d16b1d594a789f539390a44bc39f
0809 01:45:37.662644 (+ 66us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:37.662836 (+ 192us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1490587}
W20260809 01:45:37.665510 10576 catalog_manager.cc:5017] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet e155d16b1d594a789f539390a44bc39f because the tablet was not found. No further retry: Not found: Tablet not found: e155d16b1d594a789f539390a44bc39f
I20260809 01:45:37.664965 11062 leader_election.cc:304] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:37.666875 11950 tablet_replica.cc:333] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:37.668645 11950 raft_consensus.cc:2243] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:37.669109 12007 raft_consensus.cc:3060] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:37.669585 11950 raft_consensus.cc:2272] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:37.690146 11951 raft_consensus.cc:2243] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:37.690834 11951 raft_consensus.cc:2272] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:37.694912 11950 ts_tablet_manager.cc:1916] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:37.695062 11951 ts_tablet_manager.cc:1916] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:37.700107 10581 consensus_peers.cc:597] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 -> Peer 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063): Couldn't send request to peer 71bc72deb1d2402cabd817dfe3b27e88. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260809 01:45:37.684072 10591 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_636659c489a74b76ab839543e432ee5c" } modify_external_catalogs: true
I20260809 01:45:37.710232 10591 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_636659c489a74b76ab839543e432ee5c" } modify_external_catalogs: true
I20260809 01:45:37.712383 11910 raft_consensus.cc:493] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:37.712931 11910 raft_consensus.cc:515] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:37.715814 11950 ts_tablet_manager.cc:1929] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:37.716202 11950 log.cc:1199] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/2674c9dbf9e44d63b41427f2fda4e311
I20260809 01:45:37.716572 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d0a9d73d4841c393e7d2cef41fee89" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:37.716898 11950 ts_tablet_manager.cc:1950] T 2674c9dbf9e44d63b41427f2fda4e311 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:37.716202 11910 leader_election.cc:290] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
W20260809 01:45:37.718781 11283 leader_election.cc:343] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:37.722303 11950 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 79) took 1551 ms. Trace:
I20260809 01:45:37.722646 11950 rpcz_store.cc:276] 0809 01:45:36.170980 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.171159 (+ 179us) service_pool.cc:224] Handling call
0809 01:45:37.666599 (+1495440us) ts_tablet_manager.cc:1151] Deleting tablet 2674c9dbf9e44d63b41427f2fda4e311
0809 01:45:37.666668 (+ 69us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:37.707039 (+ 40371us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:37.715673 (+ 8634us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:37.722205 (+ 6532us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2982,"tablet-delete.queue_time_us":1494508}
I20260809 01:45:37.732538 11949 ts_tablet_manager.cc:1929] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:37.733068 11949 log.cc:1199] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/135d3d65ea7846d3bc2cb59d44dbedf6
I20260809 01:45:37.724054 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 2674c9dbf9e44d63b41427f2fda4e311 (table default.Table_7f7d5bf058bd48ff8e3660ca567a5718 [id=0806119aff0647e7aee1fc802147e2f5]) successfully deleted
I20260809 01:45:37.716219 11554 tablet_bootstrap.cc:492] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:37.746023 11554 ts_tablet_manager.cc:1403] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.423s user 0.012s sys 0.012s
I20260809 01:45:37.748687 11554 raft_consensus.cc:359] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:37.750588 11950 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 80) took 1577 ms. Trace:
I20260809 01:45:37.750965 11950 rpcz_store.cc:276] 0809 01:45:36.173408 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.173564 (+ 156us) service_pool.cc:224] Handling call
0809 01:45:37.750169 (+1576605us) ts_tablet_manager.cc:1151] Deleting tablet 7501d985958746bea58e0ff752d7fed8
0809 01:45:37.750234 (+ 65us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:37.750526 (+ 292us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1567558}
I20260809 01:45:37.754557 12007 raft_consensus.cc:2749] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:37.755026 11950 tablet_replica.cc:333] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:37.755851 11950 raft_consensus.cc:2243] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:37.756824 11950 raft_consensus.cc:2272] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:37.759548 11950 ts_tablet_manager.cc:1916] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:37.766687 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 7501d985958746bea58e0ff752d7fed8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7501d985958746bea58e0ff752d7fed8 already in progress: creating tablet
I20260809 01:45:37.773764 12039 tablet.cc:1724] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:37.749390 11554 raft_consensus.cc:385] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:37.785481 10592 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_f6654879d18843fd84d44ea5f170dda7" } new_table_name: "default.Table_52926e76c3ad4bbcbc425c32d4473a3c" modify_external_catalogs: true
I20260809 01:45:37.785864 11554 raft_consensus.cc:740] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:37.786602 11554 consensus_queue.cc:260] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:37.788447 11950 ts_tablet_manager.cc:1929] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:37.788558 11554 ts_tablet_manager.cc:1434] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.042s user 0.005s sys 0.001s
I20260809 01:45:37.788823 11950 log.cc:1199] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/61c83b8bcf54461bb24565e3a989100a
I20260809 01:45:37.789631 11950 ts_tablet_manager.cc:1950] T 61c83b8bcf54461bb24565e3a989100a P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:37.790040 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_ab740dbd782f44eb8f423baeede1de0f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:37.790990 11554 tablet_bootstrap.cc:492] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
W20260809 01:45:37.791674 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab740dbd782f44eb8f423baeede1de0f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:37.791786 11950 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 81) took 1610 ms. Trace:
I20260809 01:45:37.792083 11950 rpcz_store.cc:276] 0809 01:45:36.181046 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.184063 (+ 3017us) service_pool.cc:224] Handling call
0809 01:45:37.754836 (+1570773us) ts_tablet_manager.cc:1151] Deleting tablet 61c83b8bcf54461bb24565e3a989100a
0809 01:45:37.754911 (+ 75us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:37.777163 (+ 22252us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:37.788320 (+ 11157us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:37.791725 (+ 3405us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1570090}
I20260809 01:45:37.793591 11950 tablet_replica.cc:333] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:37.794396 11950 raft_consensus.cc:2243] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:37.795115 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 61c83b8bcf54461bb24565e3a989100a (table default.Table_c46005ef8c674ee4819c91c204485828 [id=8b42fb7f9413434b96d5e788d9919ec2]) successfully deleted
I20260809 01:45:37.795313 11950 raft_consensus.cc:2272] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:37.800551 11950 ts_tablet_manager.cc:1916] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:37.812305 11551 tablet_bootstrap.cc:492] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:37.812707 11551 ts_tablet_manager.cc:1403] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.665s user 0.012s sys 0.008s
I20260809 01:45:37.770972 12042 tablet.cc:1724] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:37.815156 11551 raft_consensus.cc:359] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:37.818630 11551 raft_consensus.cc:385] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:37.818924 11551 raft_consensus.cc:740] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:37.819602 11551 consensus_queue.cc:260] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:37.819672 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_f81e3edafcf34038a6081ef4cd6ab8f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:37.821202 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f81e3edafcf34038a6081ef4cd6ab8f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:37.826740 11888 raft_consensus.cc:493] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:37.827147 11888 raft_consensus.cc:515] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:37.828982 11888 leader_election.cc:290] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:37.830883 11391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb5e2ec60c24f8396daed992ca314ac" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:37.831455 11391 raft_consensus.cc:2468] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:37.832679 11062 leader_election.cc:304] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:37.833449 11888 raft_consensus.cc:2804] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:37.833801 11888 raft_consensus.cc:493] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:37.834102 11888 raft_consensus.cc:3060] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:37.840642 11551 ts_tablet_manager.cc:1434] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.028s user 0.006s sys 0.001s
I20260809 01:45:37.843103 11888 raft_consensus.cc:515] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:37.652926 11553 consensus_queue.cc:260] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:37.848780 11553 ts_tablet_manager.cc:1434] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.215s user 0.004s sys 0.004s
I20260809 01:45:37.848796 11916 raft_consensus.cc:493] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:37.852461 11916 raft_consensus.cc:515] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:37.853751 11553 tablet_bootstrap.cc:492] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
W20260809 01:45:37.854146 11063 leader_election.cc:343] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 5bb5e2ec60c24f8396daed992ca314ac
I20260809 01:45:37.855530 11916 leader_election.cc:290] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:37.863806 11551 tablet_bootstrap.cc:492] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:37.865547 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7df6343423604a00a46f203f61e98e2f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:37.867002 10804 leader_election.cc:343] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:37.870419 11554 tablet_bootstrap.cc:654] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:37.871366 11553 tablet_bootstrap.cc:654] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:37.872687 10592 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ef8a334741d3419eaa377338e0eaa889 consensus state has no leader
I20260809 01:45:37.880924 11551 tablet_bootstrap.cc:654] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:37.886348 10588 catalog_manager.cc:5719] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67). New cstate: current_term: 1 leader_uuid: "4f60c9ac9414403d9ea5a97b103c2316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } }
I20260809 01:45:37.887463 11391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb5e2ec60c24f8396daed992ca314ac" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:37.887580 10588 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233]). Expected version 2 got 0
I20260809 01:45:37.888056 11391 raft_consensus.cc:3060] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:37.894904 11391 raft_consensus.cc:2468] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
W20260809 01:45:37.895339 11063 leader_election.cc:343] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 5bb5e2ec60c24f8396daed992ca314ac
I20260809 01:45:37.896037 11062 leader_election.cc:304] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:37.896870 12007 raft_consensus.cc:2804] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:37.896775 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 75e4315b3c644846ac572883c51f23de (DEFAULT_TABLE table=default.Table_198b5459279b480eb8a7b2d0ab6f8c87 [id=850f362a7f9a4531a3d7d6de29b695ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:37.897289 12007 raft_consensus.cc:697] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:37.897871 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e4315b3c644846ac572883c51f23de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:37.898080 12007 consensus_queue.cc:237] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:37.899329 11125 tablet_service.cc:1511] Processing CreateTablet for tablet 83ac8498dff04b4e93303d7a35e62134 (DEFAULT_TABLE table=default.Table_5cf8eb3b39ee45f68e7a554f32e35f3d [id=0c687c7e332d4b238f2c85fd16c93466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:37.900409 11125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83ac8498dff04b4e93303d7a35e62134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:37.905606 11888 leader_election.cc:290] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:37.906221 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 75e4315b3c644846ac572883c51f23de (DEFAULT_TABLE table=default.Table_198b5459279b480eb8a7b2d0ab6f8c87 [id=850f362a7f9a4531a3d7d6de29b695ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:37.907160 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e4315b3c644846ac572883c51f23de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:37.911037 11950 ts_tablet_manager.cc:1929] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:37.911387 11950 log.cc:1199] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/98bee541fc1345fa83ffebeba05d8c8e
I20260809 01:45:37.912139 11950 ts_tablet_manager.cc:1950] T 98bee541fc1345fa83ffebeba05d8c8e P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:37.913185 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_82ff6d419e664eeaa2ac71124ae83342"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:37.915145 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82ff6d419e664eeaa2ac71124ae83342 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:37.915712 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 7803a6aff02d42c1833f9b2cd7332b9f
I20260809 01:45:37.922092 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 7803a6aff02d42c1833f9b2cd7332b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:37.922477 11551 tablet_bootstrap.cc:492] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:37.922878 11551 ts_tablet_manager.cc:1403] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.059s user 0.009s sys 0.000s
I20260809 01:45:37.923418 10596 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233]). Expected version 2 got 0
I20260809 01:45:37.924119 10596 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 85d0a9d73d4841c393e7d2cef41fee89 (table default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0 [id=ac1fb532e0824edaa1a3abc07758fbbe]). Expected version 1 got 0
I20260809 01:45:37.869190 11949 ts_tablet_manager.cc:1950] T 135d3d65ea7846d3bc2cb59d44dbedf6 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:37.928674 10596 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet d0091542071441e5bad0c2dca819f37f (table default.Table_fb9b15123c274d7db877eac3e7906edc [id=266a0465cb4544eba6f3e9df1eacf724]). Expected version 2 got 0
I20260809 01:45:37.930356 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 7803a6aff02d42c1833f9b2cd7332b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:37.930667 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 83ac8498dff04b4e93303d7a35e62134 (DEFAULT_TABLE table=default.Table_5cf8eb3b39ee45f68e7a554f32e35f3d [id=0c687c7e332d4b238f2c85fd16c93466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:37.930858 10873 tablet_service.cc:1511] Processing CreateTablet for tablet 75e4315b3c644846ac572883c51f23de (DEFAULT_TABLE table=default.Table_198b5459279b480eb8a7b2d0ab6f8c87 [id=850f362a7f9a4531a3d7d6de29b695ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:37.931648 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83ac8498dff04b4e93303d7a35e62134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:37.944557 11553 tablet_bootstrap.cc:492] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:37.945060 11553 ts_tablet_manager.cc:1403] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.092s user 0.010s sys 0.010s
I20260809 01:45:37.945818 11126 tablet_service.cc:1558] Processing DeleteTablet for tablet 7803a6aff02d42c1833f9b2cd7332b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:37.948047 10596 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 737d7047a0c74a03b3203112987c0cb2 (table default.Table_cf7bd2b9aec04c73b8ed72debcab795d [id=ccd0ea07da604f859047f2eca85fc566]). Expected version 2 got 0
I20260809 01:45:37.947947 11553 raft_consensus.cc:359] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:37.948760 11553 raft_consensus.cc:385] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:37.949043 11553 raft_consensus.cc:740] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:37.949940 11553 consensus_queue.cc:260] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:37.951511 11553 ts_tablet_manager.cc:1434] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
I20260809 01:45:37.957026 11951 ts_tablet_manager.cc:1929] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:37.957446 11951 log.cc:1199] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/61c83b8bcf54461bb24565e3a989100a
I20260809 01:45:37.958415 11951 ts_tablet_manager.cc:1950] T 61c83b8bcf54461bb24565e3a989100a P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:37.960822 10597 master_service.cc:960] ReplaceTablet: received request to replace tablet 737d7047a0c74a03b3203112987c0cb2 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:37.969499 10586 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233]). Expected version 2 got 0
W20260809 01:45:37.971083 10598 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2aac67e7819f4a70b1050b558868f27f consensus state has no leader
I20260809 01:45:37.971623 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 83ac8498dff04b4e93303d7a35e62134 (DEFAULT_TABLE table=default.Table_5cf8eb3b39ee45f68e7a554f32e35f3d [id=0c687c7e332d4b238f2c85fd16c93466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:37.972556 10591 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 5f8824953c794450ba274b160d891fcb
I20260809 01:45:37.973024 11126 tablet_service.cc:1511] Processing CreateTablet for tablet 36a542c773b7438ead16c5b3af224cf9 (DEFAULT_TABLE table=default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:37.974156 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36a542c773b7438ead16c5b3af224cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:37.975992 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet 5f8824953c794450ba274b160d891fcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:37.980511 11364 tablet_service.cc:1558] Processing DeleteTablet for tablet 5f8824953c794450ba274b160d891fcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:37.981659 11129 tablet_service.cc:1558] Processing DeleteTablet for tablet 5f8824953c794450ba274b160d891fcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:37.986050 11364 tablet_service.cc:1558] Processing DeleteTablet for tablet 48b7459ad0974cb9948d647459b22260 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:35 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:37.987671 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 87) took 1832 ms. Trace:
I20260809 01:45:37.988029 11951 rpcz_store.cc:276] 0809 01:45:36.155158 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.155332 (+ 174us) service_pool.cc:224] Handling call
0809 01:45:37.643625 (+1488293us) ts_tablet_manager.cc:1151] Deleting tablet 61c83b8bcf54461bb24565e3a989100a
0809 01:45:37.643728 (+ 103us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:37.811152 (+167424us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:37.956876 (+145724us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:37.987613 (+ 30737us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1472449}
I20260809 01:45:37.989223 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 61c83b8bcf54461bb24565e3a989100a (table default.Table_c46005ef8c674ee4819c91c204485828 [id=8b42fb7f9413434b96d5e788d9919ec2]) successfully deleted
I20260809 01:45:37.993721 11624 raft_consensus.cc:493] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:37.994184 11624 raft_consensus.cc:515] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:37.996030 11624 leader_election.cc:290] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:37.997510 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b51f999fab24b1ca7451ff543a124a4" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:37.999022 10804 leader_election.cc:343] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.003877 11364 tablet_service.cc:1558] Processing DeleteTablet for tablet 61c83b8bcf54461bb24565e3a989100a with delete_type TABLET_DATA_DELETED (replaced by tablet e5888017345c48c7bafe6f1c6e6945cc) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:38.009157 11553 tablet_bootstrap.cc:492] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:38.012465 11950 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 83) took 1493 ms. Trace:
I20260809 01:45:38.019505 11553 tablet_bootstrap.cc:654] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:38.019896 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 135d3d65ea7846d3bc2cb59d44dbedf6 (table default.Table_f2379f27ebbb4266ab36dafd7d80f79f [id=18e87d9e1fa3442aaa489e3abcd4640e]) successfully deleted
I20260809 01:45:38.020299 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e4315b3c644846ac572883c51f23de. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:38.020680 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 5050a590fe0240ac8c9f9191ec2475a1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5050a590fe0240ac8c9f9191ec2475a1 already in progress: creating tablet
W20260809 01:45:38.021307 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 7803a6aff02d42c1833f9b2cd7332b9f because the tablet was not found. No further retry: Not found: Tablet not found: 7803a6aff02d42c1833f9b2cd7332b9f
I20260809 01:45:38.025943 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83ac8498dff04b4e93303d7a35e62134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.026759 11364 tablet_service.cc:1511] Processing CreateTablet for tablet 36a542c773b7438ead16c5b3af224cf9 (DEFAULT_TABLE table=default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.028137 11364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36a542c773b7438ead16c5b3af224cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.029714 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 88) took 1712 ms. Trace:
I20260809 01:45:38.034152 11951 rpcz_store.cc:276] 0809 01:45:36.317021 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.317186 (+ 165us) service_pool.cc:224] Handling call
0809 01:45:38.029364 (+1712178us) ts_tablet_manager.cc:1151] Deleting tablet 7501d985958746bea58e0ff752d7fed8
0809 01:45:38.029407 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:38.029661 (+ 254us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1711388}
W20260809 01:45:38.035413 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 7501d985958746bea58e0ff752d7fed8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7501d985958746bea58e0ff752d7fed8 already in progress: creating tablet
I20260809 01:45:38.044076 10871 tablet_service.cc:1511] Processing CreateTablet for tablet 36a542c773b7438ead16c5b3af224cf9 (DEFAULT_TABLE table=default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.047179 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36a542c773b7438ead16c5b3af224cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.049160 11554 tablet_bootstrap.cc:492] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:38.049363 10588 master_service.cc:960] ReplaceTablet: received request to replace tablet ac6eded297214b58a5d8129a60ca1120 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:38.049650 11554 ts_tablet_manager.cc:1403] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.259s user 0.008s sys 0.004s
I20260809 01:45:38.051939 11624 raft_consensus.cc:493] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:38.052415 11624 raft_consensus.cc:515] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:38.054575 11624 leader_election.cc:290] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:38.055049 11951 tablet_replica.cc:333] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:38.055891 11951 raft_consensus.cc:2243] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:38.056396 11951 raft_consensus.cc:2272] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:38.056664 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc230f25baf94cbf80aa8e2771ec23ba" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:38.058471 10804 leader_election.cc:343] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.059953 11554 raft_consensus.cc:359] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.060668 11554 raft_consensus.cc:385] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:38.061029 11554 raft_consensus.cc:740] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:38.064459 10874 tablet_service.cc:1558] Processing DeleteTablet for tablet 98bee541fc1345fa83ffebeba05d8c8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:35 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:38.065205 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet 34b103e2d3854eb4bf8e5cd649cbe129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:31 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:38.067380 11949 tablet_replica.cc:333] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:38.067101 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_6f6560d974c74f2dbfe00a79fc671fcb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:38.068271 11949 raft_consensus.cc:2243] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 01:45:38.068415 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f6560d974c74f2dbfe00a79fc671fcb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:38.068912 11949 raft_consensus.cc:2272] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:38.070238 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet da86b5e4a27544c59ae6bb387440d246 with delete_type TABLET_DATA_DELETED (replaced by tablet 60d356fcc48441ca80f2df29b533404f) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:38.071631 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet b870347b07b04de5b833ab4fd3a2475f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:38.072458 11949 ts_tablet_manager.cc:1916] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:38.078202 10589 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_15f337a56a7d4b7aa19d18c0f036799c" } new_table_name: "default.Table_79500ae2591d480f93bd129b5e4e5804" modify_external_catalogs: true
I20260809 01:45:38.081931 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet 12fb852b8ea8478cb160a265459865c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 7706838afc9f4dbebc3213b82c4da126) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:38.083886 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet c795f81d2bc248eeb39b64d6aaa0370c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:45982
W20260809 01:45:38.085638 10596 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 26) took 1413 ms (client timeout 1000 ms). Trace:
W20260809 01:45:38.086769 10596 rpcz_store.cc:269] 0809 01:45:36.672580 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:37.052049 (+379469us) service_pool.cc:224] Handling call
0809 01:45:37.326336 (+274287us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:38.085579 (+759243us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":16,"spinlock_wait_cycles":279905536}
I20260809 01:45:38.088279 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_bc35fdc444ce468082e5037ebe63cf03"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:38.089666 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc35fdc444ce468082e5037ebe63cf03 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:38.012844 11950 rpcz_store.cc:276] 0809 01:45:36.433266 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.433382 (+ 116us) service_pool.cc:224] Handling call
0809 01:45:37.793391 (+1360009us) ts_tablet_manager.cc:1151] Deleting tablet 98bee541fc1345fa83ffebeba05d8c8e
0809 01:45:37.793435 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:37.851185 (+ 57750us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:37.910927 (+ 59742us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:37.926592 (+ 15665us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1359425}
I20260809 01:45:38.092715 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_50229c26ee0c44adb2b271119f7eabdf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:38.093951 11950 tablet_replica.cc:333] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
W20260809 01:45:38.094202 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_50229c26ee0c44adb2b271119f7eabdf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:38.097077 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 98bee541fc1345fa83ffebeba05d8c8e (table default.Table_e212a7cfad494f2d97ac5eada6ac15f5 [id=d2c591183b4646ca88b53698200a62c2]) successfully deleted
I20260809 01:45:38.102865 11624 raft_consensus.cc:493] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:38.103327 11624 raft_consensus.cc:515] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:38.107044 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d0a9d73d4841c393e7d2cef41fee89" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:38.107652 10910 raft_consensus.cc:2468] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:38.109575 10598 catalog_manager.cc:5719] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67). New cstate: current_term: 1 leader_uuid: "4f60c9ac9414403d9ea5a97b103c2316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: HEALTHY } } }
I20260809 01:45:38.111027 10598 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 2aac67e7819f4a70b1050b558868f27f (table default.Table_a141ed06432c4f2bad256b8a687a39e4 [id=04d4b772004b44e781c255cf271fab34]). Expected version 1 got 0
I20260809 01:45:38.111764 10598 catalog_manager.cc:5719] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67). New cstate: current_term: 1 leader_uuid: "4f60c9ac9414403d9ea5a97b103c2316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: HEALTHY } } }
I20260809 01:45:38.120667 11950 raft_consensus.cc:2243] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:37.925233 11551 raft_consensus.cc:359] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.128675 11551 raft_consensus.cc:385] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:38.130260 11551 raft_consensus.cc:740] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:38.129956 11950 raft_consensus.cc:2272] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:38.132267 11551 consensus_queue.cc:260] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.161481 11624 leader_election.cc:290] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:38.135069 11950 ts_tablet_manager.cc:1916] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:38.155385 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59de95452c0a4945beae5a9344537e0f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:38.178022 10804 leader_election.cc:343] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.176600 11126 tablet_service.cc:1511] Processing CreateTablet for tablet b84f1943c1a34e999fc7c12a5847cdb6 (DEFAULT_TABLE table=default.Table_e39314f303ee429f897d4f85b8edbf69 [id=21bc1ee96b724341904d6b0c5b10fb7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.179200 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b84f1943c1a34e999fc7c12a5847cdb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.189730 11551 ts_tablet_manager.cc:1434] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.267s user 0.007s sys 0.000s
I20260809 01:45:38.199497 11551 tablet_bootstrap.cc:492] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:38.204335 11551 tablet_bootstrap.cc:654] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:38.206374 10598 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233]). Expected version 2 got 0
I20260809 01:45:38.207093 10598 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 85d0a9d73d4841c393e7d2cef41fee89 (table default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0 [id=ac1fb532e0824edaa1a3abc07758fbbe]). Expected version 1 got 0
W20260809 01:45:38.207871 10576 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 2aac67e7819f4a70b1050b558868f27f (table default.Table_a141ed06432c4f2bad256b8a687a39e4 [id=04d4b772004b44e781c255cf271fab34], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 2aac67e7819f4a70b1050b558868f27f consensus state has no leader
I20260809 01:45:38.209232 11553 tablet_bootstrap.cc:492] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:38.209698 11553 ts_tablet_manager.cc:1403] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.201s user 0.024s sys 0.030s
I20260809 01:45:38.212235 11553 raft_consensus.cc:359] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:38.212800 11553 raft_consensus.cc:385] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:38.213032 11553 raft_consensus.cc:740] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:38.213773 11553 consensus_queue.cc:260] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:38.207197 12007 raft_consensus.cc:493] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:38.219316 11553 ts_tablet_manager.cc:1434] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.009s user 0.004s sys 0.001s
I20260809 01:45:38.218999 12007 raft_consensus.cc:515] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.220173 11553 tablet_bootstrap.cc:492] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:38.220726 12007 leader_election.cc:290] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:38.222537 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0da8d25290814e66b99c2cc1d5f8b0cf" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:38.222496 10595 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_14613c38aa0e4b258598254918b193bb" } modify_external_catalogs: true
I20260809 01:45:38.223240 10910 raft_consensus.cc:2393] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33b5652538e64ed58a09993d3188f8d7 in current term 1: Already voted for candidate 4f60c9ac9414403d9ea5a97b103c2316 in this term.
I20260809 01:45:38.223485 10595 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_14613c38aa0e4b258598254918b193bb" } modify_external_catalogs: true
I20260809 01:45:38.225256 11553 tablet_bootstrap.cc:654] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:38.233949 10588 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet ac6eded297214b58a5d8129a60ca1120
I20260809 01:45:38.235598 11129 tablet_service.cc:1558] Processing DeleteTablet for tablet ac6eded297214b58a5d8129a60ca1120 with delete_type TABLET_DATA_DELETED (replaced by tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:38.237270 10588 catalog_manager.cc:6807] ReplaceTablet: tablet ac6eded297214b58a5d8129a60ca1120 deleted and replaced by tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719
I20260809 01:45:38.238896 10588 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_334efa6e437847f9b745246d8a442b89" } new_table_name: "default.Table_c92dfb9ed1d342f197b158bef20fc0ac" modify_external_catalogs: true
I20260809 01:45:38.239123 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet ac6eded297214b58a5d8129a60ca1120 with delete_type TABLET_DATA_DELETED (replaced by tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:38.072572 11951 ts_tablet_manager.cc:1916] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:38.246059 11554 consensus_queue.cc:260] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.248736 11554 ts_tablet_manager.cc:1434] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.198s user 0.005s sys 0.000s
I20260809 01:45:38.247545 11553 tablet_bootstrap.cc:492] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:38.250200 11554 tablet_bootstrap.cc:492] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:38.249848 11553 ts_tablet_manager.cc:1403] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.004s
I20260809 01:45:38.097980 11391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7df6343423604a00a46f203f61e98e2f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:38.103196 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b51f999fab24b1ca7451ff543a124a4" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:38.254347 11949 ts_tablet_manager.cc:1929] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:38.254122 11553 raft_consensus.cc:359] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:38.254731 11553 raft_consensus.cc:385] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:38.104542 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc230f25baf94cbf80aa8e2771ec23ba" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:38.254981 11553 raft_consensus.cc:740] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
W20260809 01:45:38.118292 11289 consensus_peers.cc:597] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:38.254731 11949 log.cc:1199] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/5f8824953c794450ba274b160d891fcb
W20260809 01:45:38.255455 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 137329us
I20260809 01:45:38.255609 11553 consensus_queue.cc:260] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:38.256739 11289 leader_election.cc:304] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:38.257396 11553 ts_tablet_manager.cc:1434] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 01:45:38.165406 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59de95452c0a4945beae5a9344537e0f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:38.258214 11553 tablet_bootstrap.cc:492] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:38.258580 11976 raft_consensus.cc:2804] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:38.258927 11976 raft_consensus.cc:493] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:38.259227 11976 raft_consensus.cc:3060] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:38.261308 11551 tablet_bootstrap.cc:492] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:38.179194 11361 tablet_service.cc:1511] Processing CreateTablet for tablet b84f1943c1a34e999fc7c12a5847cdb6 (DEFAULT_TABLE table=default.Table_e39314f303ee429f897d4f85b8edbf69 [id=21bc1ee96b724341904d6b0c5b10fb7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.237779 11352 tablet_service.cc:1558] Processing DeleteTablet for tablet ac6eded297214b58a5d8129a60ca1120 with delete_type TABLET_DATA_DELETED (replaced by tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:38.261713 11551 ts_tablet_manager.cc:1403] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.062s user 0.005s sys 0.004s
I20260809 01:45:38.262226 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b84f1943c1a34e999fc7c12a5847cdb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.264343 11551 raft_consensus.cc:359] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.264950 11551 raft_consensus.cc:385] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:38.265178 11551 raft_consensus.cc:740] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:38.265708 11949 ts_tablet_manager.cc:1950] T 5f8824953c794450ba274b160d891fcb P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:38.268383 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0da8d25290814e66b99c2cc1d5f8b0cf" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:38.267055 10805 leader_election.cc:343] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.269035 10805 leader_election.cc:304] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:38.270181 10805 leader_election.cc:343] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.270141 11062 leader_election.cc:304] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:38.270531 10805 leader_election.cc:304] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:38.271734 11624 raft_consensus.cc:2749] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:38.272351 11624 raft_consensus.cc:2749] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:38.273695 12007 raft_consensus.cc:3060] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:38.276598 11976 raft_consensus.cc:515] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:38.271438 10805 leader_election.cc:343] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.278221 10805 leader_election.cc:304] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:38.279539 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 737d7047a0c74a03b3203112987c0cb2
I20260809 01:45:38.279914 10873 tablet_service.cc:1511] Processing CreateTablet for tablet b84f1943c1a34e999fc7c12a5847cdb6 (DEFAULT_TABLE table=default.Table_e39314f303ee429f897d4f85b8edbf69 [id=21bc1ee96b724341904d6b0c5b10fb7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.280844 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b84f1943c1a34e999fc7c12a5847cdb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.281421 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 737d7047a0c74a03b3203112987c0cb2 with delete_type TABLET_DATA_DELETED (replaced by tablet 325e3d19b83545a197c2e09fc9643b5e) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:38.281850 11352 tablet_service.cc:1558] Processing DeleteTablet for tablet 737d7047a0c74a03b3203112987c0cb2 with delete_type TABLET_DATA_DELETED (replaced by tablet 325e3d19b83545a197c2e09fc9643b5e) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:38.281863 12045 raft_consensus.cc:2749] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:38.282388 10597 catalog_manager.cc:6807] ReplaceTablet: tablet 737d7047a0c74a03b3203112987c0cb2 deleted and replaced by tablet 325e3d19b83545a197c2e09fc9643b5e
W20260809 01:45:38.282893 10805 leader_election.cc:343] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.283268 10805 leader_election.cc:304] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:38.284217 12045 raft_consensus.cc:2749] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:38.288587 10597 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_d1d088df5e7d452ab5a8771a8d3679dd" } modify_external_catalogs: true
I20260809 01:45:38.289170 10597 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_d1d088df5e7d452ab5a8771a8d3679dd" } modify_external_catalogs: true
I20260809 01:45:38.296136 12007 raft_consensus.cc:2749] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:38.298283 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d0a9d73d4841c393e7d2cef41fee89" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:38.298811 10910 raft_consensus.cc:3060] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:38.306067 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d0a9d73d4841c393e7d2cef41fee89" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
W20260809 01:45:38.307718 11283 leader_election.cc:343] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.309333 10910 raft_consensus.cc:2468] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:38.310779 11289 leader_election.cc:304] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:38.311097 11976 leader_election.cc:290] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:38.312003 11970 raft_consensus.cc:2804] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:38.312619 11970 raft_consensus.cc:697] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:38.313766 11970 consensus_queue.cc:237] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:38.315439 10589 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3e03033672e947eab8cc143263281133 has no consensus state
I20260809 01:45:38.316018 11950 ts_tablet_manager.cc:1929] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:38.316603 11950 log.cc:1199] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/43d84f310d8e4df7b8b2d602289025ae
I20260809 01:45:38.318300 11551 consensus_queue.cc:260] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.320585 11125 tablet_service.cc:1558] Processing DeleteTablet for tablet 737d7047a0c74a03b3203112987c0cb2 with delete_type TABLET_DATA_DELETED (replaced by tablet 325e3d19b83545a197c2e09fc9643b5e) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:38.321496 10870 tablet_service.cc:1511] Processing CreateTablet for tablet cfb9afd3be194830ba99f5e20fdaca0a (DEFAULT_TABLE table=default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.321692 11126 tablet_service.cc:1511] Processing CreateTablet for tablet cfb9afd3be194830ba99f5e20fdaca0a (DEFAULT_TABLE table=default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.323808 11553 tablet_bootstrap.cc:654] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:38.324450 11366 tablet_service.cc:1511] Processing CreateTablet for tablet cfb9afd3be194830ba99f5e20fdaca0a (DEFAULT_TABLE table=default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.325587 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfb9afd3be194830ba99f5e20fdaca0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.327682 10871 tablet_service.cc:1511] Processing CreateTablet for tablet c528b45de0d14c19af5d6836b7a14f1c (DEFAULT_TABLE table=default.Table_f2379f27ebbb4266ab36dafd7d80f79f [id=18e87d9e1fa3442aaa489e3abcd4640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.333427 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfb9afd3be194830ba99f5e20fdaca0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.338315 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c528b45de0d14c19af5d6836b7a14f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.335695 11551 ts_tablet_manager.cc:1434] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.074s user 0.005s sys 0.000s
I20260809 01:45:38.345335 11553 tablet_bootstrap.cc:492] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:38.345728 11553 ts_tablet_manager.cc:1403] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.088s user 0.005s sys 0.007s
I20260809 01:45:38.345913 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 5f8824953c794450ba274b160d891fcb (table default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]) successfully deleted
W20260809 01:45:38.346844 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 98bee541fc1345fa83ffebeba05d8c8e because the tablet was not found. No further retry: Not found: Tablet not found: 98bee541fc1345fa83ffebeba05d8c8e
I20260809 01:45:38.347937 11553 raft_consensus.cc:359] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:38.348471 11553 raft_consensus.cc:385] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 01:45:38.348445 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 34b103e2d3854eb4bf8e5cd649cbe129 because the tablet was not found. No further retry: Not found: Tablet not found: 34b103e2d3854eb4bf8e5cd649cbe129
I20260809 01:45:38.348681 11553 raft_consensus.cc:740] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:38.349254 11553 consensus_queue.cc:260] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:38.352097 11553 ts_tablet_manager.cc:1434] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
I20260809 01:45:38.352883 11553 tablet_bootstrap.cc:492] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:38.357393 11352 tablet_service.cc:1511] Processing CreateTablet for tablet 995a37a9e6b14889b876da5d76c89b84 (DEFAULT_TABLE table=default.Table_50f4db524f554499a2dab6d3bc48f053 [id=85ee30d1d77d47f691e6a0dd498d9bca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.359503 11352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 995a37a9e6b14889b876da5d76c89b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.360357 11551 tablet_bootstrap.cc:492] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:38.360958 11949 tablet_replica.cc:333] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:38.345003 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfb9afd3be194830ba99f5e20fdaca0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.362859 11125 tablet_service.cc:1511] Processing CreateTablet for tablet 995a37a9e6b14889b876da5d76c89b84 (DEFAULT_TABLE table=default.Table_50f4db524f554499a2dab6d3bc48f053 [id=85ee30d1d77d47f691e6a0dd498d9bca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.364821 11950 ts_tablet_manager.cc:1950] T 43d84f310d8e4df7b8b2d602289025ae P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:38.365592 11949 raft_consensus.cc:2243] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:38.366671 11949 raft_consensus.cc:2272] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:38.367508 11125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 995a37a9e6b14889b876da5d76c89b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.371186 11364 tablet_service.cc:1511] Processing CreateTablet for tablet c528b45de0d14c19af5d6836b7a14f1c (DEFAULT_TABLE table=default.Table_f2379f27ebbb4266ab36dafd7d80f79f [id=18e87d9e1fa3442aaa489e3abcd4640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.376106 11364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c528b45de0d14c19af5d6836b7a14f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.377081 11124 tablet_service.cc:1511] Processing CreateTablet for tablet c528b45de0d14c19af5d6836b7a14f1c (DEFAULT_TABLE table=default.Table_f2379f27ebbb4266ab36dafd7d80f79f [id=18e87d9e1fa3442aaa489e3abcd4640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.378095 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c528b45de0d14c19af5d6836b7a14f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.380002 11950 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 89) took 1549 ms. Trace:
I20260809 01:45:38.380465 11950 rpcz_store.cc:276] 0809 01:45:36.830986 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.832127 (+ 1141us) service_pool.cc:224] Handling call
0809 01:45:38.093792 (+1261665us) ts_tablet_manager.cc:1151] Deleting tablet 43d84f310d8e4df7b8b2d602289025ae
0809 01:45:38.093839 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:38.229079 (+135240us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:38.315825 (+ 86746us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:38.379910 (+ 64085us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1260780}
I20260809 01:45:38.382309 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 43d84f310d8e4df7b8b2d602289025ae (table default.Table_2712961a5fbc40689aeb5e6dcbf6d79f [id=2a521d855351481299fb19d2b1f99b37]) successfully deleted
I20260809 01:45:38.383522 11553 tablet_bootstrap.cc:654] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:38.390676 11949 ts_tablet_manager.cc:1916] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:38.382459 11950 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 91) took 1136 ms. Trace:
I20260809 01:45:38.416121 11950 rpcz_store.cc:276] 0809 01:45:37.246461 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:37.246558 (+ 97us) service_pool.cc:224] Handling call
0809 01:45:38.381963 (+1135405us) ts_tablet_manager.cc:1151] Deleting tablet 135d3d65ea7846d3bc2cb59d44dbedf6
0809 01:45:38.382022 (+ 59us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:38.382383 (+ 361us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1112496}
W20260809 01:45:38.426255 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 135d3d65ea7846d3bc2cb59d44dbedf6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 135d3d65ea7846d3bc2cb59d44dbedf6 already in progress: creating tablet
I20260809 01:45:38.426437 11551 tablet_bootstrap.cc:654] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:38.428365 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 5050a590fe0240ac8c9f9191ec2475a1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5050a590fe0240ac8c9f9191ec2475a1 already in progress: creating tablet
I20260809 01:45:38.429236 11950 tablet_replica.cc:333] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:38.430260 11950 raft_consensus.cc:2243] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:38.433192 11129 tablet_service.cc:1511] Processing CreateTablet for tablet 197b5043f6d348f89853c073b256daa4 (DEFAULT_TABLE table=default.Table_b92eb95256a74ee195ec93866ec1b673 [id=12a77217eec442c6a9199a418d9d5bb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.434154 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 197b5043f6d348f89853c073b256daa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.434075 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_3227dde355ef4460a73ac13726bf7249"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:38.434324 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 197b5043f6d348f89853c073b256daa4 (DEFAULT_TABLE table=default.Table_b92eb95256a74ee195ec93866ec1b673 [id=12a77217eec442c6a9199a418d9d5bb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.435216 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 197b5043f6d348f89853c073b256daa4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:38.435539 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3227dde355ef4460a73ac13726bf7249 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:38.399916 11951 ts_tablet_manager.cc:1929] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:38.441795 11951 log.cc:1199] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/98bee541fc1345fa83ffebeba05d8c8e
I20260809 01:45:38.442104 11553 tablet_bootstrap.cc:492] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:38.442773 11951 ts_tablet_manager.cc:1950] T 98bee541fc1345fa83ffebeba05d8c8e P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:38.443940 11553 ts_tablet_manager.cc:1403] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.091s user 0.009s sys 0.015s
I20260809 01:45:38.442708 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 995a37a9e6b14889b876da5d76c89b84 (DEFAULT_TABLE table=default.Table_50f4db524f554499a2dab6d3bc48f053 [id=85ee30d1d77d47f691e6a0dd498d9bca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.449450 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 995a37a9e6b14889b876da5d76c89b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.450188 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 91) took 1821 ms. Trace:
I20260809 01:45:38.455554 11553 raft_consensus.cc:359] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.456300 11553 raft_consensus.cc:385] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:38.456548 11553 raft_consensus.cc:740] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:38.458547 11551 tablet_bootstrap.cc:492] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:38.459002 11551 ts_tablet_manager.cc:1403] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.099s user 0.006s sys 0.005s
I20260809 01:45:38.457259 11553 consensus_queue.cc:260] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.430759 11950 raft_consensus.cc:2272] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:38.470026 11553 ts_tablet_manager.cc:1434] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.026s user 0.006s sys 0.002s
I20260809 01:45:38.470871 11553 tablet_bootstrap.cc:492] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:38.474014 11950 ts_tablet_manager.cc:1916] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:38.480743 11553 tablet_bootstrap.cc:654] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:38.482743 12045 raft_consensus.cc:493] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:38.483400 12045 raft_consensus.cc:515] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.495579 12045 leader_election.cc:290] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:38.498294 10871 tablet_service.cc:1511] Processing CreateTablet for tablet 197b5043f6d348f89853c073b256daa4 (DEFAULT_TABLE table=default.Table_b92eb95256a74ee195ec93866ec1b673 [id=12a77217eec442c6a9199a418d9d5bb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.498600 11949 ts_tablet_manager.cc:1929] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:38.499284 10586 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233]). Expected version 2 got 0
I20260809 01:45:38.501113 11949 log.cc:1199] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/da86b5e4a27544c59ae6bb387440d246
I20260809 01:45:38.502636 11949 ts_tablet_manager.cc:1950] T da86b5e4a27544c59ae6bb387440d246 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:38.504956 11553 tablet_bootstrap.cc:492] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:38.505333 11553 ts_tablet_manager.cc:1403] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.012s
I20260809 01:45:38.476791 11551 raft_consensus.cc:359] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.511122 11551 raft_consensus.cc:385] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:38.511451 11551 raft_consensus.cc:740] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:38.512101 11551 consensus_queue.cc:260] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.513433 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_001ce3fb9e1e480f915bdbd88c394244"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:38.514333 11551 ts_tablet_manager.cc:1434] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.055s user 0.004s sys 0.004s
W20260809 01:45:38.515271 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_001ce3fb9e1e480f915bdbd88c394244 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:38.518164 11553 raft_consensus.cc:359] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:38.518811 11553 raft_consensus.cc:385] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:38.519121 11553 raft_consensus.cc:740] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:38.519731 10589 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_a141ed06432c4f2bad256b8a687a39e4" } new_table_name: "default.Table_eb561ea28aa2441fb68983735558c016" modify_external_catalogs: true
I20260809 01:45:38.519861 11553 consensus_queue.cc:260] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:38.521746 11553 ts_tablet_manager.cc:1434] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.016s user 0.002s sys 0.004s
I20260809 01:45:38.527431 10592 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233]). Expected version 2 got 0
I20260809 01:45:38.527946 10592 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 85d0a9d73d4841c393e7d2cef41fee89 (table default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0 [id=ac1fb532e0824edaa1a3abc07758fbbe]). Expected version 1 got 0
I20260809 01:45:38.529054 10592 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet ef8a334741d3419eaa377338e0eaa889 (table default.Table_52926e76c3ad4bbcbc425c32d4473a3c [id=51771c6c664942e4b1df1132b5641ffa]). Expected version 1 got 0
I20260809 01:45:38.529284 11125 tablet_service.cc:1558] Processing DeleteTablet for tablet 43d84f310d8e4df7b8b2d602289025ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:30 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:38.496845 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7706838afc9f4dbebc3213b82c4da126" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:38.529920 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet 5bb5e2ec60c24f8396daed992ca314ac with delete_type TABLET_DATA_DELETED (replaced by tablet c2f86bba4bd94fbcacdc9c139e9de2d7) from {username='slave'} at 127.0.0.1:56566
W20260809 01:45:38.531128 10804 leader_election.cc:343] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.532393 11551 tablet_bootstrap.cc:492] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:38.541647 11951 rpcz_store.cc:276] 0809 01:45:36.638953 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.639562 (+ 609us) service_pool.cc:224] Handling call
0809 01:45:38.054890 (+1415328us) ts_tablet_manager.cc:1151] Deleting tablet 98bee541fc1345fa83ffebeba05d8c8e
0809 01:45:38.054931 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:38.289009 (+234078us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:38.399777 (+110768us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:38.450090 (+ 50313us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1396678}
I20260809 01:45:38.543803 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 98bee541fc1345fa83ffebeba05d8c8e (table default.Table_e212a7cfad494f2d97ac5eada6ac15f5 [id=d2c591183b4646ca88b53698200a62c2]) successfully deleted
I20260809 01:45:38.455813 11970 consensus_queue.cc:1048] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:38.409091 11554 tablet_bootstrap.cc:654] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:38.489804 11976 consensus_queue.cc:1048] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:38.493772 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7706838afc9f4dbebc3213b82c4da126" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:38.504138 11354 tablet_service.cc:1558] Processing DeleteTablet for tablet c795f81d2bc248eeb39b64d6aaa0370c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:38.511936 11910 raft_consensus.cc:493] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:38.565476 10805 leader_election.cc:343] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.565564 11910 raft_consensus.cc:515] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.566202 10805 leader_election.cc:304] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:38.567907 11910 leader_election.cc:290] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:38.567938 11624 raft_consensus.cc:2749] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:38.569552 11624 raft_consensus.cc:493] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:38.569962 11624 raft_consensus.cc:515] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.571538 11624 leader_election.cc:290] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:38.573498 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 197b5043f6d348f89853c073b256daa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.574555 11949 tablet_replica.cc:333] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
W20260809 01:45:38.591900 11283 consensus_peers.cc:597] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:38.592262 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c795f81d2bc248eeb39b64d6aaa0370c" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:38.593652 11283 leader_election.cc:343] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.594098 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b870347b07b04de5b833ab4fd3a2475f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:38.594681 11395 raft_consensus.cc:2468] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 0.
I20260809 01:45:38.600179 10595 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 6d35f291c92a45899eed0f2f10a3a1d8
I20260809 01:45:38.602630 11129 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d35f291c92a45899eed0f2f10a3a1d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:38.545850 11951 tablet_replica.cc:333] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:38.605068 11553 tablet_bootstrap.cc:492] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:38.606822 10805 leader_election.cc:304] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:38.607079 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_78c91fd32f104ad785b008e345a24e04"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:38.608438 10592 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 2aac67e7819f4a70b1050b558868f27f (table default.Table_a141ed06432c4f2bad256b8a687a39e4 [id=04d4b772004b44e781c255cf271fab34]). Expected version 1 got 0
I20260809 01:45:38.609206 10592 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet d0091542071441e5bad0c2dca819f37f (table default.Table_fb9b15123c274d7db877eac3e7906edc [id=266a0465cb4544eba6f3e9df1eacf724]). Expected version 2 got 0
I20260809 01:45:38.609330 11624 raft_consensus.cc:2804] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:38.609697 11624 raft_consensus.cc:493] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:38.610016 11624 raft_consensus.cc:3060] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:38.610143 11951 raft_consensus.cc:2243] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:38.609985 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_4d3af03d62f44239afda0e4122e17b2d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:38.610621 11951 raft_consensus.cc:2272] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:38.611527 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d3af03d62f44239afda0e4122e17b2d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:38.609189 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c795f81d2bc248eeb39b64d6aaa0370c" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
W20260809 01:45:38.608459 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_78c91fd32f104ad785b008e345a24e04 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:38.501752 11126 tablet_service.cc:1511] Processing CreateTablet for tablet 7803a6aff02d42c1833f9b2cd7332b9f (DEFAULT_TABLE table=default.Table_8a405d19ab94416c8d58fff878edae0a [id=42039f2e75f942ac8b96fb47088ad193]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.620211 10905 raft_consensus.cc:2468] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:38.621678 11289 leader_election.cc:304] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:38.622769 11910 raft_consensus.cc:2804] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:38.623078 11910 raft_consensus.cc:493] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:38.623368 11910 raft_consensus.cc:3060] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:38.618597 11950 ts_tablet_manager.cc:1929] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:38.629959 11950 log.cc:1199] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/34b103e2d3854eb4bf8e5cd649cbe129
I20260809 01:45:38.630580 11910 raft_consensus.cc:515] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.633114 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c795f81d2bc248eeb39b64d6aaa0370c" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:38.633679 10905 raft_consensus.cc:3060] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:38.634497 11950 ts_tablet_manager.cc:1950] T 34b103e2d3854eb4bf8e5cd649cbe129 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:38.635742 11624 raft_consensus.cc:515] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.639218 11950 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 93) took 1116 ms. Trace:
I20260809 01:45:38.639580 11950 rpcz_store.cc:276] 0809 01:45:37.523086 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:37.525653 (+ 2567us) service_pool.cc:224] Handling call
0809 01:45:38.429071 (+903418us) ts_tablet_manager.cc:1151] Deleting tablet 34b103e2d3854eb4bf8e5cd649cbe129
0809 01:45:38.429127 (+ 56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:38.579082 (+149955us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:38.618493 (+ 39411us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:38.639166 (+ 20673us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":902211}
I20260809 01:45:38.640930 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 34b103e2d3854eb4bf8e5cd649cbe129 (table default.Table_175d0b46158d49a88aac3833ba31533a [id=cf3f859364164b9085d525891266a825]) successfully deleted
I20260809 01:45:38.630152 11551 tablet_bootstrap.cc:654] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:38.628505 11129 tablet_service.cc:1511] Processing CreateTablet for tablet b2726cb09f734ed18bb9379f94f3f22a (DEFAULT_TABLE table=default.Table_03cfb94f08614dc5bcdaf9eb182adc1b [id=d23f238a284a47f2be2faf340b853440]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.647794 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b870347b07b04de5b833ab4fd3a2475f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:38.648164 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet da86b5e4a27544c59ae6bb387440d246 (table default.Table_29a4bd6bd10a473dbd9dbda008b9eb16 [id=67ee87fabc7749279341528fbcce16d4]) successfully deleted
I20260809 01:45:38.648314 11158 raft_consensus.cc:2468] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 0.
I20260809 01:45:38.649435 11949 raft_consensus.cc:2243] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:38.649818 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b870347b07b04de5b833ab4fd3a2475f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:38.650353 11395 raft_consensus.cc:3060] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:38.656080 11352 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d35f291c92a45899eed0f2f10a3a1d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:38.656648 11910 leader_election.cc:290] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:38.656476 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 7803a6aff02d42c1833f9b2cd7332b9f (DEFAULT_TABLE table=default.Table_8a405d19ab94416c8d58fff878edae0a [id=42039f2e75f942ac8b96fb47088ad193]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.657815 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7803a6aff02d42c1833f9b2cd7332b9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.665799 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c795f81d2bc248eeb39b64d6aaa0370c" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
W20260809 01:45:38.667093 11283 leader_election.cc:343] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.618741 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7803a6aff02d42c1833f9b2cd7332b9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.668207 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2726cb09f734ed18bb9379f94f3f22a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.671329 11553 tablet_bootstrap.cc:654] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:38.672174 10581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111) [suppressed 7 similar messages]
I20260809 01:45:38.647965 11624 leader_election.cc:290] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:38.675025 11950 tablet_replica.cc:333] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
W20260809 01:45:38.675334 10576 catalog_manager.cc:5017] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 7803a6aff02d42c1833f9b2cd7332b9f because the tablet was not found. No further retry: Not found: Tablet not found: 7803a6aff02d42c1833f9b2cd7332b9f
I20260809 01:45:38.675688 11950 raft_consensus.cc:2243] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:38.676152 11950 raft_consensus.cc:2272] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:38.677992 11395 raft_consensus.cc:2468] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 1.
I20260809 01:45:38.642416 10905 raft_consensus.cc:2468] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:38.681787 12076 raft_consensus.cc:493] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:38.683393 11289 leader_election.cc:304] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:38.683789 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 68f022e6a57a4ec0a5a8c33159e97a8e (DEFAULT_TABLE table=default.Table_5f68b1cdb1a347f78466f3333fd113ff [id=7e63b2e8084c4dc9b53d011ba1d822e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.683583 12076 raft_consensus.cc:515] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:38.684791 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68f022e6a57a4ec0a5a8c33159e97a8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.684701 11910 raft_consensus.cc:2804] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:38.685180 11910 raft_consensus.cc:697] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:38.686368 11910 consensus_queue.cc:237] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.687537 11950 ts_tablet_manager.cc:1916] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:38.689978 10805 leader_election.cc:304] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:38.690968 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 7803a6aff02d42c1833f9b2cd7332b9f (DEFAULT_TABLE table=default.Table_8a405d19ab94416c8d58fff878edae0a [id=42039f2e75f942ac8b96fb47088ad193]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.691955 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7803a6aff02d42c1833f9b2cd7332b9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.691805 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b870347b07b04de5b833ab4fd3a2475f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:38.692394 11158 raft_consensus.cc:3060] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:38.692816 10873 tablet_service.cc:1511] Processing CreateTablet for tablet b2726cb09f734ed18bb9379f94f3f22a (DEFAULT_TABLE table=default.Table_03cfb94f08614dc5bcdaf9eb182adc1b [id=d23f238a284a47f2be2faf340b853440]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.693835 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2726cb09f734ed18bb9379f94f3f22a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.694293 11951 ts_tablet_manager.cc:1916] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:38.700539 11158 raft_consensus.cc:2468] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 1.
I20260809 01:45:38.701753 12045 raft_consensus.cc:2706] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Received election callback for term 1 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260809 01:45:38.702507 11624 raft_consensus.cc:493] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:38.703209 11624 raft_consensus.cc:515] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:38.704998 11624 leader_election.cc:290] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:38.707458 10588 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 has no consensus state
I20260809 01:45:38.709340 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 4754f2e2a35f4a6a91fa333e2cf564e4
I20260809 01:45:38.715415 11949 raft_consensus.cc:2272] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:38.723287 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 4754f2e2a35f4a6a91fa333e2cf564e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:38.723639 11352 tablet_service.cc:1511] Processing CreateTablet for tablet 68f022e6a57a4ec0a5a8c33159e97a8e (DEFAULT_TABLE table=default.Table_5f68b1cdb1a347f78466f3333fd113ff [id=7e63b2e8084c4dc9b53d011ba1d822e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.726863 11352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68f022e6a57a4ec0a5a8c33159e97a8e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:38.727682 11283 consensus_peers.cc:597] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:38.754634 10597 master_service.cc:960] ReplaceTablet: received request to replace tablet 36a542c773b7438ead16c5b3af224cf9 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:38.759678 11554 tablet_bootstrap.cc:492] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:38.760094 11554 ts_tablet_manager.cc:1403] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.510s user 0.009s sys 0.005s
W20260809 01:45:38.762138 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 72270us
I20260809 01:45:38.775276 11949 ts_tablet_manager.cc:1916] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:38.779717 11361 tablet_service.cc:1511] Processing CreateTablet for tablet b2726cb09f734ed18bb9379f94f3f22a (DEFAULT_TABLE table=default.Table_03cfb94f08614dc5bcdaf9eb182adc1b [id=d23f238a284a47f2be2faf340b853440]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.781144 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2726cb09f734ed18bb9379f94f3f22a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.786183 10598 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet d0091542071441e5bad0c2dca819f37f (table default.Table_fb9b15123c274d7db877eac3e7906edc [id=266a0465cb4544eba6f3e9df1eacf724]). Expected version 2 got 0
I20260809 01:45:38.786978 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 4754f2e2a35f4a6a91fa333e2cf564e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:38.790705 12076 leader_election.cc:290] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:38.795644 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 68f022e6a57a4ec0a5a8c33159e97a8e (DEFAULT_TABLE table=default.Table_5f68b1cdb1a347f78466f3333fd113ff [id=7e63b2e8084c4dc9b53d011ba1d822e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.799252 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d35f291c92a45899eed0f2f10a3a1d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:38.798652 10598 master_service.cc:960] ReplaceTablet: received request to replace tablet 201329c676ec4b6abbb72b22519bc41d from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:38.800251 11553 tablet_bootstrap.cc:492] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:38.800612 11553 ts_tablet_manager.cc:1403] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.196s user 0.014s sys 0.004s
I20260809 01:45:38.808410 10591 catalog_manager.cc:5719] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67). New cstate: current_term: 1 leader_uuid: "4f60c9ac9414403d9ea5a97b103c2316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: HEALTHY } } }
I20260809 01:45:38.810537 10591 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 85d0a9d73d4841c393e7d2cef41fee89 (table default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0 [id=ac1fb532e0824edaa1a3abc07758fbbe]). Expected version 1 got 0
I20260809 01:45:38.811290 10591 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233]). Expected version 2 got 0
I20260809 01:45:38.811276 11553 raft_consensus.cc:359] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:38.811852 11553 raft_consensus.cc:385] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:38.812104 11553 raft_consensus.cc:740] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:38.701912 11551 tablet_bootstrap.cc:492] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:38.813740 11551 ts_tablet_manager.cc:1403] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.282s user 0.011s sys 0.000s
I20260809 01:45:38.816581 11551 raft_consensus.cc:359] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.817178 11551 raft_consensus.cc:385] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:38.715342 11125 tablet_service.cc:1558] Processing DeleteTablet for tablet 4754f2e2a35f4a6a91fa333e2cf564e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:38.817548 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68f022e6a57a4ec0a5a8c33159e97a8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.817404 11551 raft_consensus.cc:740] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:38.818941 11551 consensus_queue.cc:260] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.821934 11551 ts_tablet_manager.cc:1434] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260809 01:45:38.822721 11124 tablet_service.cc:1558] Processing DeleteTablet for tablet b870347b07b04de5b833ab4fd3a2475f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:38.795764 11157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cef88ffe9e95437da81793795742def3" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:38.822659 10589 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_84e4a5312bbb43c9bd562d0f742537e2" } modify_external_catalogs: true
I20260809 01:45:38.796116 11123 tablet_service.cc:1511] Processing CreateTablet for tablet e3684fa56f9549f492a96a5c765eda78 (DEFAULT_TABLE table=default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.823253 10589 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_84e4a5312bbb43c9bd562d0f742537e2" } modify_external_catalogs: true
W20260809 01:45:38.825810 10804 leader_election.cc:343] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.829471 11123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3684fa56f9549f492a96a5c765eda78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.831014 11950 ts_tablet_manager.cc:1929] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260809 01:45:38.835711 10581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.125:42063, user_credentials={real_user=slave}} blocked reactor thread for 163544us
I20260809 01:45:38.831357 11950 log.cc:1199] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/5f8824953c794450ba274b160d891fcb
I20260809 01:45:38.842048 11950 ts_tablet_manager.cc:1950] T 5f8824953c794450ba274b160d891fcb P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:38.845031 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 5f8824953c794450ba274b160d891fcb (table default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]) successfully deleted
I20260809 01:45:38.822902 11551 tablet_bootstrap.cc:492] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
W20260809 01:45:38.850939 11063 consensus_peers.cc:597] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5bb5e2ec60c24f8396daed992ca314ac. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:38.851344 10586 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 2aac67e7819f4a70b1050b558868f27f (table default.Table_eb561ea28aa2441fb68983735558c016 [id=04d4b772004b44e781c255cf271fab34]). Expected version 2 got 0
I20260809 01:45:38.788398 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af35cf31a66474cb0575b2fb1547616" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:38.854190 10804 leader_election.cc:343] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:38.866681 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet ac6eded297214b58a5d8129a60ca1120 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac6eded297214b58a5d8129a60ca1120 already in progress: creating tablet
I20260809 01:45:38.812696 11553 consensus_queue.cc:260] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:38.873047 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 737d7047a0c74a03b3203112987c0cb2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 737d7047a0c74a03b3203112987c0cb2 already in progress: creating tablet
I20260809 01:45:38.873155 11624 raft_consensus.cc:493] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:38.873574 11624 raft_consensus.cc:515] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:38.874783 10576 catalog_manager.cc:5017] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 43d84f310d8e4df7b8b2d602289025ae because the tablet was not found. No further retry: Not found: Tablet not found: 43d84f310d8e4df7b8b2d602289025ae
I20260809 01:45:38.875972 11624 leader_election.cc:290] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:38.877002 11553 ts_tablet_manager.cc:1434] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.076s user 0.007s sys 0.000s
I20260809 01:45:38.877925 11553 tablet_bootstrap.cc:492] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:38.883921 11950 tablet_replica.cc:333] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:38.884721 11950 raft_consensus.cc:2243] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:38.886782 10873 tablet_service.cc:1511] Processing CreateTablet for tablet e3684fa56f9549f492a96a5c765eda78 (DEFAULT_TABLE table=default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.774015 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af35cf31a66474cb0575b2fb1547616" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:38.887815 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3684fa56f9549f492a96a5c765eda78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.891369 11950 pending_rounds.cc:70] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7: Trying to abort 1 pending ops.
I20260809 01:45:38.891636 11950 pending_rounds.cc:77] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438785715900416 op_type: NO_OP noop_request { }
I20260809 01:45:38.891685 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cef88ffe9e95437da81793795742def3" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:38.891942 11950 raft_consensus.cc:2889] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:38.892242 11551 tablet_bootstrap.cc:654] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:38.892261 11950 raft_consensus.cc:2272] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:38.893296 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d35f291c92a45899eed0f2f10a3a1d8" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:38.896188 11950 ts_tablet_manager.cc:1916] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:38.900568 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 36a542c773b7438ead16c5b3af224cf9
I20260809 01:45:38.902005 10597 catalog_manager.cc:6807] ReplaceTablet: tablet 36a542c773b7438ead16c5b3af224cf9 deleted and replaced by tablet a38bd344108649ed8e1687f82154000b
I20260809 01:45:38.909017 11129 tablet_service.cc:1558] Processing DeleteTablet for tablet 36a542c773b7438ead16c5b3af224cf9 with delete_type TABLET_DATA_DELETED (replaced by tablet a38bd344108649ed8e1687f82154000b) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:38.895730 12074 consensus_queue.cc:1048] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:38.911285 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet 36a542c773b7438ead16c5b3af224cf9 with delete_type TABLET_DATA_DELETED (replaced by tablet a38bd344108649ed8e1687f82154000b) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:38.911234 10591 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_eb561ea28aa2441fb68983735558c016" } new_table_name: "default.Table_7f2585a3c363483ea2b745cde05d5e18" modify_external_catalogs: true
I20260809 01:45:38.911924 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d35f291c92a45899eed0f2f10a3a1d8" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:38.914129 10805 leader_election.cc:343] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.914731 10805 leader_election.cc:304] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:38.916031 10597 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_5f4d617a86c2487fa9f64cb58f14de3c" } modify_external_catalogs: true
W20260809 01:45:38.917757 10804 leader_election.cc:343] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.919528 11366 tablet_service.cc:1511] Processing CreateTablet for tablet e3684fa56f9549f492a96a5c765eda78 (DEFAULT_TABLE table=default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:38.920717 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3684fa56f9549f492a96a5c765eda78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:38.921983 10597 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_5f4d617a86c2487fa9f64cb58f14de3c" } modify_external_catalogs: true
W20260809 01:45:38.935016 10805 leader_election.cc:343] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.935408 10805 leader_election.cc:304] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:38.936491 10805 leader_election.cc:343] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:38.936882 10805 leader_election.cc:304] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:38.937676 12095 raft_consensus.cc:2749] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:38.938323 12095 raft_consensus.cc:2749] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:38.938875 12095 raft_consensus.cc:2749] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:38.949191 11949 ts_tablet_manager.cc:1929] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:38.949636 11949 log.cc:1199] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/b870347b07b04de5b833ab4fd3a2475f
I20260809 01:45:38.950407 11949 ts_tablet_manager.cc:1950] T b870347b07b04de5b833ab4fd3a2475f P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:38.952939 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet b870347b07b04de5b833ab4fd3a2475f (table default.Table_9590b2028c934cbb91c44edf337bb871 [id=a0eb60025e8c435e958f40f1ec5c366c]) successfully deleted
I20260809 01:45:38.918555 11554 raft_consensus.cc:359] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.957589 11554 raft_consensus.cc:385] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:38.957901 11554 raft_consensus.cc:740] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:38.969074 11551 tablet_bootstrap.cc:492] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:38.969666 11551 ts_tablet_manager.cc:1403] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.147s user 0.008s sys 0.008s
W20260809 01:45:38.976790 11743 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:45:38.977175 11743 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
I20260809 01:45:38.977135 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_198b5459279b480eb8a7b2d0ab6f8c87" } new_table_name: "default.Table_d32d2e398c0f4c548296bb53315f1c05" modify_external_catalogs: true
W20260809 01:45:38.977562 11743 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:45:38.977928 11743 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:45:38.977864 11551 raft_consensus.cc:359] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.982800 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_2b3bf023a9304fa982b5bc6f1a0b701b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:38.984185 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b3bf023a9304fa982b5bc6f1a0b701b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:38.985611 11551 raft_consensus.cc:385] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:38.985888 11551 raft_consensus.cc:740] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:38.986052 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 0da8d25290814e66b99c2cc1d5f8b0cf
I20260809 01:45:38.986507 11551 consensus_queue.cc:260] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:38.989122 10598 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 201329c676ec4b6abbb72b22519bc41d
I20260809 01:45:38.989974 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_7168034c2262465e932139aa19fd6fd4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:38.994120 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7168034c2262465e932139aa19fd6fd4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:38.990803 10598 catalog_manager.cc:6807] ReplaceTablet: tablet 201329c676ec4b6abbb72b22519bc41d deleted and replaced by tablet 8baa4ae62edf46b5a386661a6826bbab
I20260809 01:45:38.998459 11126 tablet_service.cc:1558] Processing DeleteTablet for tablet 201329c676ec4b6abbb72b22519bc41d with delete_type TABLET_DATA_DELETED (replaced by tablet 8baa4ae62edf46b5a386661a6826bbab) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:38.999115 11123 tablet_service.cc:1558] Processing DeleteTablet for tablet 0da8d25290814e66b99c2cc1d5f8b0cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:39.007869 10598 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 85d0a9d73d4841c393e7d2cef41fee89 (table default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0 [id=ac1fb532e0824edaa1a3abc07758fbbe]). Expected version 1 got 0
I20260809 01:45:39.010263 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet 0da8d25290814e66b99c2cc1d5f8b0cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:39.010944 11551 ts_tablet_manager.cc:1434] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.041s user 0.004s sys 0.002s
I20260809 01:45:39.011582 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet 201329c676ec4b6abbb72b22519bc41d with delete_type TABLET_DATA_DELETED (replaced by tablet 8baa4ae62edf46b5a386661a6826bbab) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:39.020077 11553 tablet_bootstrap.cc:654] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:39.023222 11551 tablet_bootstrap.cc:492] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:39.026544 10598 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 85d0a9d73d4841c393e7d2cef41fee89 (table default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0 [id=ac1fb532e0824edaa1a3abc07758fbbe]). Expected version 1 got 0
I20260809 01:45:39.029239 11950 ts_tablet_manager.cc:1929] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:39.029722 11950 log.cc:1199] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/5bb5e2ec60c24f8396daed992ca314ac
I20260809 01:45:39.031544 11950 ts_tablet_manager.cc:1950] T 5bb5e2ec60c24f8396daed992ca314ac P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:39.037825 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_ce7493c5349e41d890087f1336689d4a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:39.039338 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce7493c5349e41d890087f1336689d4a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:39.039484 11554 consensus_queue.cc:260] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.042726 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 5bb5e2ec60c24f8396daed992ca314ac (table default.Table_5f4d617a86c2487fa9f64cb58f14de3c [id=f64daa5a2706436081e9166a5f3a0cd7]) successfully deleted
I20260809 01:45:39.043356 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_920e3c9eef6241a3bbe3da8898a42132"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:39.044528 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 6d35f291c92a45899eed0f2f10a3a1d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d35f291c92a45899eed0f2f10a3a1d8 already in progress: creating tablet
W20260809 01:45:39.041489 11364 consensus_meta.cc:287] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.553s user 0.004s sys 0.008s
W20260809 01:45:39.044911 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_920e3c9eef6241a3bbe3da8898a42132 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:39.046018 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 4754f2e2a35f4a6a91fa333e2cf564e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4754f2e2a35f4a6a91fa333e2cf564e4 already in progress: creating tablet
I20260809 01:45:39.047657 11950 tablet_replica.cc:333] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:39.048250 11950 raft_consensus.cc:2243] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:39.048648 11950 raft_consensus.cc:2272] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:39.056524 12076 raft_consensus.cc:493] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:39.057396 12076 raft_consensus.cc:515] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:39.058961 11949 tablet_replica.cc:333] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:39.059963 11949 raft_consensus.cc:2243] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:39.060586 11949 raft_consensus.cc:2272] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:39.061429 12007 raft_consensus.cc:493] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:39.062038 12007 raft_consensus.cc:515] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.062665 11950 ts_tablet_manager.cc:1916] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:39.064667 12007 leader_election.cc:290] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:39.065800 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0091542071441e5bad0c2dca819f37f" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:39.066619 10910 raft_consensus.cc:2468] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:39.068747 11063 leader_election.cc:304] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:39.065145 12007 raft_consensus.cc:493] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:39.083200 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16897857df24c6da5744a99d5f687c0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:39.085394 10804 leader_election.cc:343] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:39.088073 12113 raft_consensus.cc:2804] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:39.088847 11949 ts_tablet_manager.cc:1916] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:39.094996 12076 leader_election.cc:290] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:39.070420 12007 raft_consensus.cc:515] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.103616 12007 leader_election.cc:290] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:39.104409 12113 raft_consensus.cc:493] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:39.103698 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d0a9d73d4841c393e7d2cef41fee89" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:39.105821 10910 raft_consensus.cc:2393] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33b5652538e64ed58a09993d3188f8d7 in current term 1: Already voted for candidate 4f60c9ac9414403d9ea5a97b103c2316 in this term.
I20260809 01:45:39.121755 12113 raft_consensus.cc:3060] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:39.113951 11551 tablet_bootstrap.cc:654] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:39.127343 12076 raft_consensus.cc:493] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:39.127730 12076 raft_consensus.cc:515] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:39.129601 12007 raft_consensus.cc:493] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:39.129415 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 86079.8us
I20260809 01:45:39.130153 12007 raft_consensus.cc:515] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.130793 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16897857df24c6da5744a99d5f687c0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:39.132467 10805 leader_election.cc:343] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:39.132917 12007 leader_election.cc:290] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:39.133111 10805 leader_election.cc:304] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:39.134689 11624 raft_consensus.cc:2749] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:39.136349 11554 ts_tablet_manager.cc:1434] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.376s user 0.003s sys 0.008s
I20260809 01:45:39.136148 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f86bba4bd94fbcacdc9c139e9de2d7" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:39.137429 11554 tablet_bootstrap.cc:492] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:39.137949 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aac67e7819f4a70b1050b558868f27f" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:39.140231 12076 leader_election.cc:290] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:39.141871 10804 leader_election.cc:343] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:39.143797 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f86bba4bd94fbcacdc9c139e9de2d7" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:39.145742 10805 leader_election.cc:343] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:39.145998 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0091542071441e5bad0c2dca819f37f" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:39.146216 11553 tablet_bootstrap.cc:492] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:39.146659 11553 ts_tablet_manager.cc:1403] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.269s user 0.006s sys 0.007s
I20260809 01:45:39.146251 10805 leader_election.cc:304] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:39.148274 12076 raft_consensus.cc:2749] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:39.148494 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d0a9d73d4841c393e7d2cef41fee89" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:39.150916 11553 raft_consensus.cc:359] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:39.151495 11553 raft_consensus.cc:385] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:39.151484 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aac67e7819f4a70b1050b558868f27f" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:39.151731 11553 raft_consensus.cc:740] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
W20260809 01:45:39.153743 11062 leader_election.cc:343] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:39.155953 11062 leader_election.cc:304] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:39.156896 12007 raft_consensus.cc:3060] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:39.166790 12007 raft_consensus.cc:2749] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:39.173089 11553 consensus_queue.cc:260] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:39.169044 11062 leader_election.cc:304] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:39.177977 11551 tablet_bootstrap.cc:492] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:39.178438 11551 ts_tablet_manager.cc:1403] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.155s user 0.005s sys 0.009s
I20260809 01:45:39.181331 12120 raft_consensus.cc:2749] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:39.184295 11553 ts_tablet_manager.cc:1434] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.037s user 0.006s sys 0.002s
I20260809 01:45:39.185806 12116 tablet.cc:1724] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:39.191568 10598 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_29a4bd6bd10a473dbd9dbda008b9eb16" } modify_external_catalogs: true
I20260809 01:45:39.192101 10598 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_29a4bd6bd10a473dbd9dbda008b9eb16" } modify_external_catalogs: true
I20260809 01:45:39.195639 11553 tablet_bootstrap.cc:492] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:39.218540 10588 master_service.cc:960] ReplaceTablet: received request to replace tablet dc230f25baf94cbf80aa8e2771ec23ba from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:39.219939 11551 raft_consensus.cc:359] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.220556 11551 raft_consensus.cc:385] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:39.220782 11551 raft_consensus.cc:740] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:39.221760 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet b870347b07b04de5b833ab4fd3a2475f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:39.223021 11950 ts_tablet_manager.cc:1929] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:39.223356 11950 log.cc:1199] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/b870347b07b04de5b833ab4fd3a2475f
I20260809 01:45:39.192772 12113 raft_consensus.cc:515] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.234714 12113 leader_election.cc:290] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:39.236846 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0091542071441e5bad0c2dca819f37f" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:39.237375 10910 raft_consensus.cc:3060] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:39.238124 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet da86b5e4a27544c59ae6bb387440d246 with delete_type TABLET_DATA_DELETED (replaced by tablet 60d356fcc48441ca80f2df29b533404f) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:39.239272 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet 5f8824953c794450ba274b160d891fcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:39.235420 11950 ts_tablet_manager.cc:1950] T b870347b07b04de5b833ab4fd3a2475f P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:39.244022 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet b870347b07b04de5b833ab4fd3a2475f (table default.Table_9590b2028c934cbb91c44edf337bb871 [id=a0eb60025e8c435e958f40f1ec5c366c]) successfully deleted
I20260809 01:45:39.244069 10910 raft_consensus.cc:2468] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:39.235808 11551 consensus_queue.cc:260] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:39.244850 10579 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 98bee541fc1345fa83ffebeba05d8c8e (table default.Table_e212a7cfad494f2d97ac5eada6ac15f5 [id=d2c591183b4646ca88b53698200a62c2], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 98bee541fc1345fa83ffebeba05d8c8e consensus state has no leader
I20260809 01:45:39.245775 11063 leader_election.cc:304] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:39.246650 12007 raft_consensus.cc:2804] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:39.246531 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0091542071441e5bad0c2dca819f37f" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:39.247054 12007 raft_consensus.cc:697] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:39.247964 12007 consensus_queue.cc:237] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:39.250041 11062 leader_election.cc:343] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:39.257748 12115 tablet.cc:1724] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260809 01:45:39.262557 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 36a542c773b7438ead16c5b3af224cf9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36a542c773b7438ead16c5b3af224cf9 already in progress: creating tablet
I20260809 01:45:39.262791 11551 ts_tablet_manager.cc:1434] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.084s user 0.007s sys 0.001s
W20260809 01:45:39.264139 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 201329c676ec4b6abbb72b22519bc41d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 201329c676ec4b6abbb72b22519bc41d already in progress: creating tablet
I20260809 01:45:39.271703 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet 637f0d6261f041b5aeff5fcc709a3dfc with delete_type TABLET_DATA_DELETED (replaced by tablet 995a37a9e6b14889b876da5d76c89b84) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:39.275820 11551 tablet_bootstrap.cc:492] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:39.275951 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet 135d3d65ea7846d3bc2cb59d44dbedf6 with delete_type TABLET_DATA_DELETED (replaced by tablet c528b45de0d14c19af5d6836b7a14f1c) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:39.280658 11551 tablet_bootstrap.cc:654] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:39.280856 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet c2f86bba4bd94fbcacdc9c139e9de2d7
I20260809 01:45:39.281450 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet 7501d985958746bea58e0ff752d7fed8 with delete_type TABLET_DATA_DELETED (replaced by tablet b2726cb09f734ed18bb9379f94f3f22a) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:39.285683 11950 tablet_replica.cc:333] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:39.286537 11123 tablet_service.cc:1558] Processing DeleteTablet for tablet c2f86bba4bd94fbcacdc9c139e9de2d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:39.287185 11950 raft_consensus.cc:2243] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:39.287704 11950 raft_consensus.cc:2272] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:39.289937 11950 ts_tablet_manager.cc:1916] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:39.303840 10586 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 75e4315b3c644846ac572883c51f23de consensus state has no leader
I20260809 01:45:39.304517 11950 ts_tablet_manager.cc:1929] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:39.304929 11950 log.cc:1199] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/0da8d25290814e66b99c2cc1d5f8b0cf
I20260809 01:45:39.305850 11950 ts_tablet_manager.cc:1950] T 0da8d25290814e66b99c2cc1d5f8b0cf P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:39.307461 12113 raft_consensus.cc:493] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:39.307819 12113 raft_consensus.cc:515] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.309638 12113 leader_election.cc:290] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:39.311484 10595 master_service.cc:960] ReplaceTablet: received request to replace tablet a9d1ed215ba644abbae09e78016b2c21 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:39.311640 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c795f81d2bc248eeb39b64d6aaa0370c" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:39.312387 10910 raft_consensus.cc:2393] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33b5652538e64ed58a09993d3188f8d7 in current term 1: Already voted for candidate 4f60c9ac9414403d9ea5a97b103c2316 in this term.
I20260809 01:45:39.318393 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet a9d1ed215ba644abbae09e78016b2c21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:39.318598 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 0da8d25290814e66b99c2cc1d5f8b0cf (table default.Table_84e4a5312bbb43c9bd562d0f742537e2 [id=5cc4859efd27438a8570acaafe1e6e78]) successfully deleted
W20260809 01:45:39.319195 10592 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 27) took 1241 ms (client timeout 1000 ms). Trace:
W20260809 01:45:39.319509 10592 rpcz_store.cc:269] 0809 01:45:38.077716 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.079382 (+ 1666us) service_pool.cc:224] Handling call
0809 01:45:39.319122 (+1239740us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":28}
W20260809 01:45:39.320695 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet c2f86bba4bd94fbcacdc9c139e9de2d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c2f86bba4bd94fbcacdc9c139e9de2d7 already in progress: creating tablet
I20260809 01:45:39.329177 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet c2f86bba4bd94fbcacdc9c139e9de2d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:39.338708 11949 ts_tablet_manager.cc:1929] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:39.350287 11554 tablet_bootstrap.cc:654] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:39.362442 11970 raft_consensus.cc:493] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:39.363268 11970 raft_consensus.cc:515] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.365211 11949 log.cc:1199] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/12fb852b8ea8478cb160a265459865c4
I20260809 01:45:39.366006 11949 ts_tablet_manager.cc:1950] T 12fb852b8ea8478cb160a265459865c4 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:39.368054 11970 leader_election.cc:290] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:39.369679 11970 raft_consensus.cc:493] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:39.370127 11970 raft_consensus.cc:515] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.364787 11553 tablet_bootstrap.cc:654] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:39.375774 11283 leader_election.cc:343] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: 34b103e2d3854eb4bf8e5cd649cbe129
I20260809 01:45:39.376561 11910 consensus_queue.cc:1048] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:39.376155 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737d7047a0c74a03b3203112987c0cb2" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:39.378165 11283 leader_election.cc:343] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:39.360200 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c795f81d2bc248eeb39b64d6aaa0370c" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:39.379477 11289 leader_election.cc:343] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 34b103e2d3854eb4bf8e5cd649cbe129
I20260809 01:45:39.380156 11289 leader_election.cc:304] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:39.381374 11910 raft_consensus.cc:2749] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:45:39.383242 11062 leader_election.cc:304] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:39.385892 12113 raft_consensus.cc:3060] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:39.395743 11551 tablet_bootstrap.cc:492] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:39.396121 11551 ts_tablet_manager.cc:1403] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.120s user 0.029s sys 0.009s
I20260809 01:45:39.359207 11951 ts_tablet_manager.cc:1929] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:39.398159 12113 raft_consensus.cc:2749] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:39.378242 11970 leader_election.cc:290] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:39.400442 11551 raft_consensus.cc:359] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.401077 11551 raft_consensus.cc:385] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:39.401422 11551 raft_consensus.cc:740] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:39.402338 11551 consensus_queue.cc:260] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.404826 11551 ts_tablet_manager.cc:1434] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260809 01:45:39.407015 11364 tablet_service.cc:1558] Processing DeleteTablet for tablet b870347b07b04de5b833ab4fd3a2475f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:39.407851 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 0da8d25290814e66b99c2cc1d5f8b0cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:39.410403 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet 201329c676ec4b6abbb72b22519bc41d with delete_type TABLET_DATA_DELETED (replaced by tablet 8baa4ae62edf46b5a386661a6826bbab) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:39.398440 11951 log.cc:1199] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/43d84f310d8e4df7b8b2d602289025ae
I20260809 01:45:39.407761 11352 tablet_service.cc:1558] Processing DeleteTablet for tablet 36a542c773b7438ead16c5b3af224cf9 with delete_type TABLET_DATA_DELETED (replaced by tablet a38bd344108649ed8e1687f82154000b) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:39.370546 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 113) took 1294 ms. Trace:
I20260809 01:45:39.416139 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737d7047a0c74a03b3203112987c0cb2" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:39.416934 10910 raft_consensus.cc:2468] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:39.417591 11551 tablet_bootstrap.cc:492] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:39.420140 12076 raft_consensus.cc:493] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:39.420660 12076 raft_consensus.cc:515] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:39.424500 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5050a590fe0240ac8c9f9191ec2475a1" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:39.426276 12076 leader_election.cc:290] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:39.427925 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5050a590fe0240ac8c9f9191ec2475a1" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:39.427390 10804 leader_election.cc:343] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:39.431603 10805 leader_election.cc:343] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:39.431941 10805 leader_election.cc:304] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:39.432848 12076 raft_consensus.cc:2749] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:39.434957 11551 tablet_bootstrap.cc:654] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:39.424069 11289 leader_election.cc:304] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:39.440033 11970 raft_consensus.cc:2804] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:39.440538 11970 raft_consensus.cc:493] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:39.441254 11970 raft_consensus.cc:3060] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:39.448333 12103 consensus_queue.cc:1048] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:39.414266 11949 rpcz_store.cc:276] 0809 01:45:38.076464 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.081692 (+ 5228us) service_pool.cc:224] Handling call
0809 01:45:39.058785 (+977093us) ts_tablet_manager.cc:1151] Deleting tablet 12fb852b8ea8478cb160a265459865c4
0809 01:45:39.058843 (+ 58us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:39.163371 (+104528us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:39.338578 (+175207us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:39.370460 (+ 31882us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":976291}
I20260809 01:45:39.455461 11951 ts_tablet_manager.cc:1950] T 43d84f310d8e4df7b8b2d602289025ae P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:39.464217 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 93) took 2776 ms. Trace:
I20260809 01:45:39.472352 11551 tablet_bootstrap.cc:492] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:39.478909 11551 ts_tablet_manager.cc:1403] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.061s user 0.019s sys 0.009s
I20260809 01:45:39.481765 11551 raft_consensus.cc:359] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.482389 11551 raft_consensus.cc:385] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:39.482637 11551 raft_consensus.cc:740] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:39.483315 11551 consensus_queue.cc:260] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.487756 11551 ts_tablet_manager.cc:1434] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20260809 01:45:39.488644 11551 tablet_bootstrap.cc:492] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:39.479686 11554 tablet_bootstrap.cc:492] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:39.529779 11949 tablet_replica.cc:333] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:39.529742 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 12fb852b8ea8478cb160a265459865c4 (table default.Table_57ef64be310d4250b77b141860e944d5 [id=212edfae1bf849c3b46674ec191ee76d]) successfully deleted
I20260809 01:45:39.529877 11554 ts_tablet_manager.cc:1403] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.393s user 0.013s sys 0.005s
I20260809 01:45:39.530666 11949 raft_consensus.cc:2243] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:39.479884 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet c2f86bba4bd94fbcacdc9c139e9de2d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:39.496984 11970 raft_consensus.cc:515] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.533042 11554 raft_consensus.cc:359] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.534613 11554 raft_consensus.cc:385] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:39.534947 11554 raft_consensus.cc:740] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:39.535686 11554 consensus_queue.cc:260] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.536187 11970 leader_election.cc:290] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:39.536489 11910 raft_consensus.cc:493] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:39.537084 11910 raft_consensus.cc:515] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.538130 11554 ts_tablet_manager.cc:1434] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20260809 01:45:39.538676 11951 rpcz_store.cc:276] 0809 01:45:36.701509 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:36.703912 (+ 2403us) service_pool.cc:224] Handling call
0809 01:45:38.544595 (+1840683us) ts_tablet_manager.cc:1151] Deleting tablet 43d84f310d8e4df7b8b2d602289025ae
0809 01:45:38.544666 (+ 71us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:39.115185 (+570519us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:39.359066 (+243881us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:39.464149 (+105083us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4578,"tablet-delete.queue_time_us":1714900}
I20260809 01:45:39.544152 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 43d84f310d8e4df7b8b2d602289025ae (table default.Table_2712961a5fbc40689aeb5e6dcbf6d79f [id=2a521d855351481299fb19d2b1f99b37]) successfully deleted
I20260809 01:45:39.544672 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737d7047a0c74a03b3203112987c0cb2" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:39.545768 11158 raft_consensus.cc:3060] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:39.541356 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 99) took 2300 ms. Trace:
I20260809 01:45:39.551134 11949 raft_consensus.cc:2272] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:39.551999 10588 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet dc230f25baf94cbf80aa8e2771ec23ba
I20260809 01:45:39.555433 11123 tablet_service.cc:1558] Processing DeleteTablet for tablet dc230f25baf94cbf80aa8e2771ec23ba with delete_type TABLET_DATA_DELETED (replaced by tablet 19fe81e277cd4c798a07ceafa0b60566) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:39.550424 11951 rpcz_store.cc:276] 0809 01:45:37.240919 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:37.241065 (+ 146us) service_pool.cc:224] Handling call
0809 01:45:39.540505 (+2299440us) ts_tablet_manager.cc:1151] Deleting tablet 135d3d65ea7846d3bc2cb59d44dbedf6
0809 01:45:39.540855 (+ 350us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:39.541277 (+ 422us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2250440}
I20260809 01:45:39.556622 11951 tablet_replica.cc:333] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
W20260809 01:45:39.557083 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet dc230f25baf94cbf80aa8e2771ec23ba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dc230f25baf94cbf80aa8e2771ec23ba already in progress: creating tablet
W20260809 01:45:39.558234 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 135d3d65ea7846d3bc2cb59d44dbedf6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 135d3d65ea7846d3bc2cb59d44dbedf6 already in progress: creating tablet
I20260809 01:45:39.558933 10588 catalog_manager.cc:6807] ReplaceTablet: tablet dc230f25baf94cbf80aa8e2771ec23ba deleted and replaced by tablet 19fe81e277cd4c798a07ceafa0b60566
I20260809 01:45:39.558974 11554 tablet_bootstrap.cc:492] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:39.560272 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet dc230f25baf94cbf80aa8e2771ec23ba with delete_type TABLET_DATA_DELETED (replaced by tablet 19fe81e277cd4c798a07ceafa0b60566) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:39.562875 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12fb852b8ea8478cb160a265459865c4" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:39.558959 11951 raft_consensus.cc:2243] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:39.563714 11156 raft_consensus.cc:2468] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:39.564625 11951 raft_consensus.cc:2272] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:39.564535 11910 leader_election.cc:290] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:39.565963 11283 leader_election.cc:304] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:39.569036 10595 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_50f4db524f554499a2dab6d3bc48f053" } modify_external_catalogs: true
I20260809 01:45:39.570866 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_d29473576bc0470bbea9f4ef1fb00e8b" } new_table_name: "default.Table_db89b9afbd84498e98b8c8fb48f0fd5d" modify_external_catalogs: true
I20260809 01:45:39.571015 10595 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_50f4db524f554499a2dab6d3bc48f053" } modify_external_catalogs: true
I20260809 01:45:39.571768 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet dc230f25baf94cbf80aa8e2771ec23ba with delete_type TABLET_DATA_DELETED (replaced by tablet 19fe81e277cd4c798a07ceafa0b60566) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:39.569193 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_67456cecac004439bf596d7f3a46d602"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:39.574537 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737d7047a0c74a03b3203112987c0cb2" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
W20260809 01:45:39.574566 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_67456cecac004439bf596d7f3a46d602 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:39.575052 10905 raft_consensus.cc:3060] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:39.570159 10592 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_49db2b3dc5474004946e08a5245687c3" } new_table_name: "default.Table_0bcdcc37d5f04f40ab4dd02b66baf4a6" modify_external_catalogs: true
I20260809 01:45:39.577615 12076 raft_consensus.cc:493] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:39.578245 12076 raft_consensus.cc:515] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:39.580859 12076 leader_election.cc:290] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:39.585403 11123 tablet_service.cc:1558] Processing DeleteTablet for tablet c795f81d2bc248eeb39b64d6aaa0370c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:39.586128 11950 tablet_replica.cc:333] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:39.586270 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6159d660d544b3e907a08372027c0e2" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:39.586810 11950 raft_consensus.cc:2243] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:39.587198 11950 raft_consensus.cc:2272] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:39.529250 11551 tablet_bootstrap.cc:654] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:39.588413 10804 leader_election.cc:343] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:39.588850 11949 ts_tablet_manager.cc:1916] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:39.589267 11950 ts_tablet_manager.cc:1916] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:39.593058 11553 tablet_bootstrap.cc:492] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:39.593495 11158 raft_consensus.cc:2468] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:39.601997 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_cf4d3615b483433d81b9cb61a945a25e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:39.611620 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cf4d3615b483433d81b9cb61a945a25e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:39.615692 11743 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.8.84:40957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.8.126:42157,127.10.8.125:42063,127.10.8.124:33929
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.125:42063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.8.125
--webserver_port=46803
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 01:43:22 UTC on e7f111948823
build id 13251
TSAN enabled
I20260809 01:45:39.617915 11743 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:45:39.620994 11743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 01:45:39.593495 11553 ts_tablet_manager.cc:1403] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.408s user 0.002s sys 0.009s
I20260809 01:45:39.634066 11950 ts_tablet_manager.cc:1929] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:39.634532 11950 log.cc:1199] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/c795f81d2bc248eeb39b64d6aaa0370c
I20260809 01:45:39.635310 11950 ts_tablet_manager.cc:1950] T c795f81d2bc248eeb39b64d6aaa0370c P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:39.638230 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet c795f81d2bc248eeb39b64d6aaa0370c (table default.Table_4eef5dd402bc41168cd7b02c32e84835 [id=ff7d5da48c94470296b8c7154d68388c]) successfully deleted
W20260809 01:45:39.658324 12100 log.cc:927] Time spent T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316: Append to log took a long time: real 0.090s user 0.000s sys 0.000s
I20260809 01:45:39.658202 11283 leader_election.cc:304] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:39.659317 11951 ts_tablet_manager.cc:1916] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:39.664921 12144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:45:39.657545 11970 raft_consensus.cc:2804] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:39.678026 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6159d660d544b3e907a08372027c0e2" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:39.665001 11910 raft_consensus.cc:2804] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:39.679077 11910 raft_consensus.cc:697] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:39.680341 11910 consensus_queue.cc:237] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:39.682796 10805 leader_election.cc:343] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:39.683413 10805 leader_election.cc:304] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:39.684312 12076 raft_consensus.cc:2749] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:39.664757 11289 leader_election.cc:343] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 12fb852b8ea8478cb160a265459865c4
I20260809 01:45:39.695067 11553 raft_consensus.cc:359] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.695804 11553 raft_consensus.cc:385] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:39.696110 11553 raft_consensus.cc:740] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:39.696851 11553 consensus_queue.cc:260] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:39.701771 10581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.125:42063, user_credentials={real_user=slave}} blocked reactor thread for 178847us
I20260809 01:45:39.666738 11970 raft_consensus.cc:493] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:39.706907 11970 raft_consensus.cc:3060] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:39.715027 10905 raft_consensus.cc:2468] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
W20260809 01:45:39.719025 11283 consensus_peers.cc:597] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c795f81d2bc248eeb39b64d6aaa0370c. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:39.725301 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_eb6df7e248d44109898ab900441bda8b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:39.726723 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb6df7e248d44109898ab900441bda8b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:39.729557 11554 tablet_bootstrap.cc:654] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:39.732198 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_608518db25c1495388ad65d72dcbe817"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:39.733593 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_608518db25c1495388ad65d72dcbe817 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:39.747210 11158 raft_consensus.cc:3060] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:39.761953 12097 mvcc.cc:204] Tried to move back new op lower bound from 7316438789109415936 to 7316438784054935552. Current Snapshot: MvccSnapshot[applied={T|T < 7316438789109415936}]
I20260809 01:45:39.784353 12115 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:53476 (request call id 121) took 1128 ms. Trace:
I20260809 01:45:39.790251 11553 ts_tablet_manager.cc:1434] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.165s user 0.006s sys 0.001s
I20260809 01:45:39.791126 11553 tablet_bootstrap.cc:492] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:39.797187 10596 master_service.cc:960] ReplaceTablet: received request to replace tablet 44ac6aecc93447f7935fbd656fd2a86b from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:39.808900 10599 master_service.cc:960] ReplaceTablet: received request to replace tablet 7501d985958746bea58e0ff752d7fed8 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:39.819984 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_6e83741f1d33450c91d06322e7b2ef98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:39.821688 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6e83741f1d33450c91d06322e7b2ef98 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:39.785027 12115 rpcz_store.cc:276] 0809 01:45:38.656237 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.656333 (+ 96us) service_pool.cc:224] Handling call
0809 01:45:39.784267 (+1127934us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:38.746510 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0809 01:45:38.747234 (+ 724us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 01:45:38.747282 (+ 48us) alter_schema_op.cc:66] Acquired schema lock
0809 01:45:38.747369 (+ 87us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0809 01:45:38.747494 (+ 125us) alter_schema_op.cc:128] START. Timestamp: 1786239938747416
0809 01:45:38.747522 (+ 28us) op_driver.cc:348] REPLICATION: starting
0809 01:45:38.748343 (+ 821us) log.cc:844] Serialized 173 byte log entry
0809 01:45:39.099408 (+351065us) spinlock_profiling.cc:243] Waited 46.1 ms on lock 0x7b18000c5e00. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0eeae45c01 00007f0eeae52581 00007f0ee1f50d33 00007f0ee1f53121 00007f0ee1f530d9 00007f0ee1f53069 00007f0ee1f53031 00007f0ee1f5232d 00000000005bd8c4 00000000005bd6f9 00007f0ee1f341ea
0809 01:45:39.136469 (+ 37061us) op_driver.cc:464] REPLICATION: finished
0809 01:45:39.198043 (+ 61574us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0809 01:45:39.345628 (+147585us) log.cc:844] Serialized 5 byte log entry
0809 01:45:39.549142 (+203514us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:39.561752 (+ 12610us) log.cc:844] Serialized 14 byte log entry
0809 01:45:39.561969 (+ 217us) alter_schema_op.cc:73] Released schema lock
0809 01:45:39.657129 (+ 95160us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
0809 01:45:39.784107 (+126978us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b5400080628. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0eeb412c21 00007f0eeb41188c 00007f0ee953beed 00007f0ee953bd8a 00007f0ee953bc37 00007f0ee9543d69 00007f0ee9543533 00007f0ee9547741 00007f0ee95476f9 00007f0ee9547689 00007f0ee9547651
Metrics: {"child_traces":[["op",{"apply.queue_time_us":61330,"mutex_wait_us":42119,"prepare.queue_time_us":66912,"prepare.run_cpu_time_us":37363,"prepare.run_wall_time_us":297225,"raft.queue_time_us":19222,"raft.run_cpu_time_us":28,"raft.run_wall_time_us":29,"replication_time_us":388814,"spinlock_wait_cycles":392796160,"thread_start_us":467188,"threads_started":3,"wal-append.queue_time_us":284142}]]}
I20260809 01:45:39.831732 12076 raft_consensus.cc:493] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:39.832412 12076 raft_consensus.cc:515] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.834983 12076 leader_election.cc:290] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:39.836746 11157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d356fcc48441ca80f2df29b533404f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:39.838146 10804 leader_election.cc:343] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:39.838598 10596 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 44ac6aecc93447f7935fbd656fd2a86b
I20260809 01:45:39.840463 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet 44ac6aecc93447f7935fbd656fd2a86b with delete_type TABLET_DATA_DELETED (replaced by tablet a04881d1c5254053bef50dd3d9d475f2) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:39.840795 10596 catalog_manager.cc:6807] ReplaceTablet: tablet 44ac6aecc93447f7935fbd656fd2a86b deleted and replaced by tablet a04881d1c5254053bef50dd3d9d475f2
I20260809 01:45:39.841359 11123 tablet_service.cc:1558] Processing DeleteTablet for tablet 44ac6aecc93447f7935fbd656fd2a86b with delete_type TABLET_DATA_DELETED (replaced by tablet a04881d1c5254053bef50dd3d9d475f2) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:39.842136 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet 44ac6aecc93447f7935fbd656fd2a86b with delete_type TABLET_DATA_DELETED (replaced by tablet a04881d1c5254053bef50dd3d9d475f2) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:39.843077 11950 tablet_replica.cc:333] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:39.844182 11950 raft_consensus.cc:2243] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:39.844861 11950 raft_consensus.cc:2272] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:39.847050 11950 ts_tablet_manager.cc:1916] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:39.847101 12115 tablet.cc:1724] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:39.864933 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_e2eb9de1f32b4341ab1745468556e4b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:39.866402 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e2eb9de1f32b4341ab1745468556e4b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:39.871006 12168 tablet.cc:1724] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:39.883164 11950 ts_tablet_manager.cc:1929] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:45:39.887001 11970 raft_consensus.cc:515] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.885406 11950 log.cc:1199] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/44ac6aecc93447f7935fbd656fd2a86b
I20260809 01:45:39.889192 11950 ts_tablet_manager.cc:1950] T 44ac6aecc93447f7935fbd656fd2a86b P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:39.914947 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 44ac6aecc93447f7935fbd656fd2a86b (table default.Table_57234dd818414b7397d1ac621bdb06a7 [id=17cbc8bf77e142d08d03bd7a051e00ad]) successfully deleted
I20260809 01:45:39.956955 12171 tablet.cc:1724] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:39.978284 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_d9c4bfa0bd764f46b7e9026317ed8e57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:39.979736 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d9c4bfa0bd764f46b7e9026317ed8e57 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:39.982332 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d356fcc48441ca80f2df29b533404f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:39.982744 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_bf875ac0c8864c968aeb0483b1d7836f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:39.984167 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf875ac0c8864c968aeb0483b1d7836f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:39.986333 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 128191us
W20260809 01:45:39.987542 11289 consensus_peers.cc:597] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:39.996662 12076 raft_consensus.cc:493] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:39.997068 12076 raft_consensus.cc:515] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:39.999557 12076 leader_election.cc:290] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:40.000134 11553 tablet_bootstrap.cc:654] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:40.004967 10805 leader_election.cc:343] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:40.008931 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501d985958746bea58e0ff752d7fed8" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:40.010275 11949 ts_tablet_manager.cc:1929] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:40.010694 11949 log.cc:1199] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/c795f81d2bc248eeb39b64d6aaa0370c
I20260809 01:45:40.011405 11949 ts_tablet_manager.cc:1950] T c795f81d2bc248eeb39b64d6aaa0370c P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:40.005446 10805 leader_election.cc:304] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:40.026840 12076 raft_consensus.cc:2749] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:40.037983 12160 tablet.cc:1724] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260809 01:45:40.044109 10805 leader_election.cc:343] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:40.049842 12007 consensus_queue.cc:1048] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:40.055068 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_a600cb6fbde0453695e637643d0f71e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:40.058413 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a600cb6fbde0453695e637643d0f71e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:40.059141 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 114) took 1975 ms. Trace:
I20260809 01:45:40.062098 11553 tablet_bootstrap.cc:492] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:40.062842 11553 ts_tablet_manager.cc:1403] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.272s user 0.011s sys 0.009s
I20260809 01:45:40.066795 11553 raft_consensus.cc:359] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:40.067432 11553 raft_consensus.cc:385] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:40.067667 11553 raft_consensus.cc:740] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:40.068286 11553 consensus_queue.cc:260] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:40.060844 11949 rpcz_store.cc:276] 0809 01:45:38.083469 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.083619 (+ 150us) service_pool.cc:224] Handling call
0809 01:45:39.529629 (+1446010us) ts_tablet_manager.cc:1151] Deleting tablet c795f81d2bc248eeb39b64d6aaa0370c
0809 01:45:39.529669 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:39.860059 (+330390us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:40.010102 (+150043us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:40.058728 (+ 48626us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21046,"tablet-delete.queue_time_us":1445304}
I20260809 01:45:40.073050 11553 ts_tablet_manager.cc:1434] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260809 01:45:40.073436 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet c795f81d2bc248eeb39b64d6aaa0370c (table default.Table_4eef5dd402bc41168cd7b02c32e84835 [id=ff7d5da48c94470296b8c7154d68388c]) successfully deleted
I20260809 01:45:40.075222 11553 tablet_bootstrap.cc:492] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:40.081770 11949 tablet_replica.cc:333] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:40.087649 11554 tablet_bootstrap.cc:492] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:40.086671 11949 raft_consensus.cc:2243] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:40.088074 11554 ts_tablet_manager.cc:1403] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.529s user 0.017s sys 0.004s
I20260809 01:45:40.090530 11949 raft_consensus.cc:2272] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:40.092356 10588 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_085fd4536b524eb9aacb353d95ffa166"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:40.102161 11949 ts_tablet_manager.cc:1916] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:40.101964 10588 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_085fd4536b524eb9aacb353d95ffa166 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:40.109476 11554 raft_consensus.cc:359] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:40.110637 11554 raft_consensus.cc:385] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:40.110937 11554 raft_consensus.cc:740] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:40.112026 11554 consensus_queue.cc:260] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:40.117854 12129 mvcc.cc:204] Tried to move back new op lower bound from 7316438792446558208 to 7316438784054935552. Current Snapshot: MvccSnapshot[applied={T|T < 7316438792446558208}]
I20260809 01:45:40.117241 11553 tablet_bootstrap.cc:654] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:40.123543 12171 tablet.cc:1724] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:40.131400 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501d985958746bea58e0ff752d7fed8" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:40.143904 11553 tablet_bootstrap.cc:492] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:40.144361 11553 ts_tablet_manager.cc:1403] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.069s user 0.011s sys 0.008s
I20260809 01:45:40.153856 11553 raft_consensus.cc:359] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:40.154644 11553 raft_consensus.cc:385] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:40.154937 11553 raft_consensus.cc:740] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:40.158196 11553 consensus_queue.cc:260] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:40.164359 11551 tablet_bootstrap.cc:492] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:40.164847 11551 ts_tablet_manager.cc:1403] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.676s user 0.027s sys 0.038s
W20260809 01:45:40.166838 10804 leader_election.cc:343] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:40.167352 10804 leader_election.cc:304] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:40.168855 12076 raft_consensus.cc:2749] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:40.180781 11553 ts_tablet_manager.cc:1434] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.036s user 0.007s sys 0.007s
I20260809 01:45:40.181994 11553 tablet_bootstrap.cc:492] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:40.210470 11553 tablet_bootstrap.cc:654] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:40.211620 12186 raft_consensus.cc:493] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:40.215451 11389 raft_consensus.cc:3060] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:40.212414 12186 raft_consensus.cc:515] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:40.221698 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0d79e03e66410487a72f1b47b327a0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:40.228195 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_39b1a27949b04026a729a622750b6b57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:40.229889 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_39b1a27949b04026a729a622750b6b57 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:40.232380 10804 leader_election.cc:343] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:40.235512 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0d79e03e66410487a72f1b47b327a0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:40.236956 11949 ts_tablet_manager.cc:1929] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:40.239459 11554 ts_tablet_manager.cc:1434] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.151s user 0.005s sys 0.005s
I20260809 01:45:40.241145 11554 tablet_bootstrap.cc:492] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:40.251787 11970 leader_election.cc:290] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
W20260809 01:45:40.237437 10805 leader_election.cc:343] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:40.254272 10805 leader_election.cc:304] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:40.255224 12076 raft_consensus.cc:2749] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:40.238327 12186 leader_election.cc:290] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:40.246517 11553 tablet_bootstrap.cc:492] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
W20260809 01:45:40.258436 11289 leader_election.cc:343] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 12fb852b8ea8478cb160a265459865c4
I20260809 01:45:40.258773 11553 ts_tablet_manager.cc:1403] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.077s user 0.010s sys 0.016s
I20260809 01:45:40.262902 11553 raft_consensus.cc:359] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:40.263829 11553 raft_consensus.cc:385] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:40.264156 11553 raft_consensus.cc:740] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:40.265121 11553 consensus_queue.cc:260] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:40.269726 11949 log.cc:1199] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/ac6eded297214b58a5d8129a60ca1120
I20260809 01:45:40.272955 11949 ts_tablet_manager.cc:1950] T ac6eded297214b58a5d8129a60ca1120 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:40.273814 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 115) took 2035 ms. Trace:
I20260809 01:45:40.274148 11949 rpcz_store.cc:276] 0809 01:45:38.238889 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.238994 (+ 105us) service_pool.cc:224] Handling call
0809 01:45:40.076908 (+1837914us) ts_tablet_manager.cc:1151] Deleting tablet ac6eded297214b58a5d8129a60ca1120
0809 01:45:40.081654 (+ 4746us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:40.169009 (+ 87355us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:40.236843 (+ 67834us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:40.273743 (+ 36900us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1837201}
I20260809 01:45:40.275316 11949 tablet_replica.cc:333] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:40.275570 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet ac6eded297214b58a5d8129a60ca1120 (table default.Table_c92dfb9ed1d342f197b158bef20fc0ac [id=96595f8bb6b84744bf7fc74f074a51ba]) successfully deleted
I20260809 01:45:40.275969 11949 raft_consensus.cc:2243] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:40.276372 11949 raft_consensus.cc:2272] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:40.281750 11553 ts_tablet_manager.cc:1434] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.023s user 0.005s sys 0.004s
I20260809 01:45:40.286087 11553 tablet_bootstrap.cc:492] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:40.300904 11949 ts_tablet_manager.cc:1916] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:40.319043 11553 tablet_bootstrap.cc:654] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:40.320801 10576 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 12fb852b8ea8478cb160a265459865c4 (table default.Table_57ef64be310d4250b77b141860e944d5 [id=212edfae1bf849c3b46674ec191ee76d], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 12fb852b8ea8478cb160a265459865c4 consensus state has no leader
I20260809 01:45:40.339376 11123 tablet_service.cc:1511] Processing CreateTablet for tablet b9493892c1ff48e7a0558ff0fcb7a0f3 (DEFAULT_TABLE table=default.Table_50229c26ee0c44adb2b271119f7eabdf [id=59cbeb450bbc4fb99011118dbfe5809a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.344225 10873 tablet_service.cc:1511] Processing CreateTablet for tablet b9493892c1ff48e7a0558ff0fcb7a0f3 (DEFAULT_TABLE table=default.Table_50229c26ee0c44adb2b271119f7eabdf [id=59cbeb450bbc4fb99011118dbfe5809a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.345319 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9493892c1ff48e7a0558ff0fcb7a0f3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:40.356518 10804 consensus_peers.cc:597] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 44ac6aecc93447f7935fbd656fd2a86b. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:40.357477 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 2692a931500443d298e25a2b362b22a1 (DEFAULT_TABLE table=default.Table_82ff6d419e664eeaa2ac71124ae83342 [id=295bede819e9407e8a7e6a861d1b36ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.358369 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2692a931500443d298e25a2b362b22a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.362387 10871 tablet_service.cc:1511] Processing CreateTablet for tablet 91941616dfdd4df3aaa7c8460876d273 (DEFAULT_TABLE table=default.Table_ab740dbd782f44eb8f423baeede1de0f [id=92630f3e58394023a931a5e2c40e8591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.364195 10869 tablet_service.cc:1511] Processing CreateTablet for tablet c3bf43978dc94c2b9a7f3c18a4677c40 (DEFAULT_TABLE table=default.Table_e03575a4e6bb46afb0346fad51b99eb5 [id=97f7c7d3e84a4032887e35cf5ea63006]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.370162 11129 tablet_service.cc:1511] Processing CreateTablet for tablet 91941616dfdd4df3aaa7c8460876d273 (DEFAULT_TABLE table=default.Table_ab740dbd782f44eb8f423baeede1de0f [id=92630f3e58394023a931a5e2c40e8591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.370224 11126 tablet_service.cc:1511] Processing CreateTablet for tablet 2692a931500443d298e25a2b362b22a1 (DEFAULT_TABLE table=default.Table_82ff6d419e664eeaa2ac71124ae83342 [id=295bede819e9407e8a7e6a861d1b36ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.371484 10874 tablet_service.cc:1511] Processing CreateTablet for tablet 403cb09c1786491ab0d40502038bf9c0 (DEFAULT_TABLE table=default.Table_63e5e407241d4eaea93f27d8285ab33a [id=9e066a2a246442b78bfc003520f4cfbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.372536 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 403cb09c1786491ab0d40502038bf9c0 (DEFAULT_TABLE table=default.Table_63e5e407241d4eaea93f27d8285ab33a [id=9e066a2a246442b78bfc003520f4cfbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.372726 11127 tablet_service.cc:1511] Processing CreateTablet for tablet c3bf43978dc94c2b9a7f3c18a4677c40 (DEFAULT_TABLE table=default.Table_e03575a4e6bb46afb0346fad51b99eb5 [id=97f7c7d3e84a4032887e35cf5ea63006]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.373831 11125 tablet_service.cc:1511] Processing CreateTablet for tablet 3e03033672e947eab8cc143263281133 (DEFAULT_TABLE table=default.Table_79500ae2591d480f93bd129b5e4e5804 [id=24d2e675d969421fba4647380880f606]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.376009 10874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 403cb09c1786491ab0d40502038bf9c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.378185 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91941616dfdd4df3aaa7c8460876d273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.382376 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3bf43978dc94c2b9a7f3c18a4677c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.372074 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233]). Expected version 2 got 0
I20260809 01:45:40.384860 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 85d0a9d73d4841c393e7d2cef41fee89 (table default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0 [id=ac1fb532e0824edaa1a3abc07758fbbe]). Expected version 1 got 0
I20260809 01:45:40.387944 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet ef8a334741d3419eaa377338e0eaa889 (table default.Table_52926e76c3ad4bbcbc425c32d4473a3c [id=51771c6c664942e4b1df1132b5641ffa]). Expected version 1 got 0
I20260809 01:45:40.388664 11553 tablet_bootstrap.cc:492] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:40.389213 11553 ts_tablet_manager.cc:1403] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.103s user 0.007s sys 0.007s
I20260809 01:45:40.393177 11553 raft_consensus.cc:359] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:40.389335 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 2aac67e7819f4a70b1050b558868f27f (table default.Table_7f2585a3c363483ea2b745cde05d5e18 [id=04d4b772004b44e781c255cf271fab34]). Expected version 3 got 0
I20260809 01:45:40.393951 11553 raft_consensus.cc:385] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:40.394207 11553 raft_consensus.cc:740] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:40.396978 11949 ts_tablet_manager.cc:1929] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:40.398172 11949 log.cc:1199] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/737d7047a0c74a03b3203112987c0cb2
I20260809 01:45:40.399360 11120 tablet_service.cc:1511] Processing CreateTablet for tablet a38bd344108649ed8e1687f82154000b (DEFAULT_TABLE table=default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.397501 10877 tablet_service.cc:1511] Processing CreateTablet for tablet a38bd344108649ed8e1687f82154000b (DEFAULT_TABLE table=default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.397217 10867 tablet_service.cc:1511] Processing CreateTablet for tablet 3e03033672e947eab8cc143263281133 (DEFAULT_TABLE table=default.Table_79500ae2591d480f93bd129b5e4e5804 [id=24d2e675d969421fba4647380880f606]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.403841 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38bd344108649ed8e1687f82154000b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.404636 10867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e03033672e947eab8cc143263281133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.403048 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet c16897857df24c6da5744a99d5f687c0 (table default.Table_6d4e0b3afc864450bdf1a8fb35f7dc75 [id=9b307bc98a124b07a2f46fafb20ce1f0]). Expected version 1 got 0
I20260809 01:45:40.414193 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet d0091542071441e5bad0c2dca819f37f (table default.Table_fb9b15123c274d7db877eac3e7906edc [id=266a0465cb4544eba6f3e9df1eacf724]). Expected version 2 got 0
I20260809 01:45:40.405357 11949 ts_tablet_manager.cc:1950] T 737d7047a0c74a03b3203112987c0cb2 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:40.414837 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_57ef64be310d4250b77b141860e944d5 [id=212edfae1bf849c3b46674ec191ee76d]). Expected version 2 got 0
W20260809 01:45:40.420651 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8baa4ae62edf46b5a386661a6826bbab has no consensus state
I20260809 01:45:40.421401 10597 master_service.cc:960] ReplaceTablet: received request to replace tablet c2f86bba4bd94fbcacdc9c139e9de2d7 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:40.423004 10866 tablet_service.cc:1511] Processing CreateTablet for tablet 0354e4c7bf354f22a9c3c389e0e8ec92 (DEFAULT_TABLE table=default.Table_f81e3edafcf34038a6081ef4cd6ab8f3 [id=e1db9d7b43aa44e6b9a6e3ca7098c95f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.425931 10866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0354e4c7bf354f22a9c3c389e0e8ec92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.427718 10589 catalog_manager.cc:5719] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66). New cstate: current_term: 1 leader_uuid: "33b5652538e64ed58a09993d3188f8d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:40.429117 10589 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet d0091542071441e5bad0c2dca819f37f (table default.Table_fb9b15123c274d7db877eac3e7906edc [id=266a0465cb4544eba6f3e9df1eacf724]). Expected version 2 got 0
I20260809 01:45:40.429819 10589 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 85d0a9d73d4841c393e7d2cef41fee89 (table default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0 [id=ac1fb532e0824edaa1a3abc07758fbbe]). Expected version 1 got 0
I20260809 01:45:40.434614 11119 tablet_service.cc:1511] Processing CreateTablet for tablet 0354e4c7bf354f22a9c3c389e0e8ec92 (DEFAULT_TABLE table=default.Table_f81e3edafcf34038a6081ef4cd6ab8f3 [id=e1db9d7b43aa44e6b9a6e3ca7098c95f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.435951 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 117) took 2154 ms. Trace:
I20260809 01:45:40.436307 11949 rpcz_store.cc:276] 0809 01:45:38.281169 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.281308 (+ 139us) service_pool.cc:224] Handling call
0809 01:45:40.275189 (+1993881us) ts_tablet_manager.cc:1151] Deleting tablet 737d7047a0c74a03b3203112987c0cb2
0809 01:45:40.275227 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:40.347581 (+ 72354us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:40.387219 (+ 39638us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:40.435882 (+ 48663us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1993366}
I20260809 01:45:40.444864 11118 tablet_service.cc:1511] Processing CreateTablet for tablet cd09bea5263344919ce26593fe958547 (DEFAULT_TABLE table=default.Table_6f6560d974c74f2dbfe00a79fc671fcb [id=cb35a16a1e1241abb696b413e3c1d8cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.448077 11117 tablet_service.cc:1511] Processing CreateTablet for tablet 761370ec8b164fd184a2e091030f610b (DEFAULT_TABLE table=default.Table_4d3af03d62f44239afda0e4122e17b2d [id=3e8bf3dc52ab433b84eeafb2c2685e21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.448732 10865 tablet_service.cc:1511] Processing CreateTablet for tablet cd09bea5263344919ce26593fe958547 (DEFAULT_TABLE table=default.Table_6f6560d974c74f2dbfe00a79fc671fcb [id=cb35a16a1e1241abb696b413e3c1d8cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.449339 11949 tablet_replica.cc:333] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:40.452399 11949 raft_consensus.cc:2243] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:40.452337 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 737d7047a0c74a03b3203112987c0cb2 (table default.Table_cf7bd2b9aec04c73b8ed72debcab795d [id=ccd0ea07da604f859047f2eca85fc566]) successfully deleted
I20260809 01:45:40.450950 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd09bea5263344919ce26593fe958547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.453987 11949 raft_consensus.cc:2272] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:40.458801 10864 tablet_service.cc:1511] Processing CreateTablet for tablet 761370ec8b164fd184a2e091030f610b (DEFAULT_TABLE table=default.Table_4d3af03d62f44239afda0e4122e17b2d [id=3e8bf3dc52ab433b84eeafb2c2685e21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.467203 11116 tablet_service.cc:1511] Processing CreateTablet for tablet 7c7a511512184f53921cea081f82aa37 (DEFAULT_TABLE table=default.Table_bc35fdc444ce468082e5037ebe63cf03 [id=e566d88eebc84e5284e7c632ae9b6fa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.468704 10864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 761370ec8b164fd184a2e091030f610b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.473922 11553 consensus_queue.cc:260] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:40.476617 11361 tablet_service.cc:1511] Processing CreateTablet for tablet b9493892c1ff48e7a0558ff0fcb7a0f3 (DEFAULT_TABLE table=default.Table_50229c26ee0c44adb2b271119f7eabdf [id=59cbeb450bbc4fb99011118dbfe5809a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.477583 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9493892c1ff48e7a0558ff0fcb7a0f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.482342 11115 tablet_service.cc:1511] Processing CreateTablet for tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 (DEFAULT_TABLE table=default.Table_c92dfb9ed1d342f197b158bef20fc0ac [id=96595f8bb6b84744bf7fc74f074a51ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.491969 11951 ts_tablet_manager.cc:1929] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:40.492492 11951 log.cc:1199] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/34b103e2d3854eb4bf8e5cd649cbe129
I20260809 01:45:40.499110 11114 tablet_service.cc:1511] Processing CreateTablet for tablet afb6c244108a4ff3a2df0585e5280294 (DEFAULT_TABLE table=default.Table_f84f34b0360549f39d13b657f425307c [id=96a100a09e6d4375ad92065772eba726]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.505290 11113 tablet_service.cc:1511] Processing CreateTablet for tablet bb7a8b6d06064addb02979a7feedf756 (DEFAULT_TABLE table=default.Table_001ce3fb9e1e480f915bdbd88c394244 [id=43255c746b5c4fcc91454fcad2c801a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.510267 11112 tablet_service.cc:1511] Processing CreateTablet for tablet 9fffa48ccf58476da79dccd9c008b67c (DEFAULT_TABLE table=default.Table_3227dde355ef4460a73ac13726bf7249 [id=3e51bbb9edf142ecabc9c92777fd80b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.524055 11949 ts_tablet_manager.cc:1916] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:40.521054 11111 tablet_service.cc:1511] Processing CreateTablet for tablet 325e3d19b83545a197c2e09fc9643b5e (DEFAULT_TABLE table=default.Table_cf7bd2b9aec04c73b8ed72debcab795d [id=ccd0ea07da604f859047f2eca85fc566]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.527922 11110 tablet_service.cc:1511] Processing CreateTablet for tablet e93334cc766044e79d37f293bc8556fb (DEFAULT_TABLE table=default.Table_2b01d8bf86d44f96b64752f511b02f50 [id=7cede7d9f9bf44f997b90481537918dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.532181 11553 ts_tablet_manager.cc:1434] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.143s user 0.006s sys 0.000s
I20260809 01:45:40.533061 11553 tablet_bootstrap.cc:492] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:40.482405 12186 raft_consensus.cc:493] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:40.547039 12186 raft_consensus.cc:515] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:40.546988 10862 tablet_service.cc:1511] Processing CreateTablet for tablet 7c7a511512184f53921cea081f82aa37 (DEFAULT_TABLE table=default.Table_bc35fdc444ce468082e5037ebe63cf03 [id=e566d88eebc84e5284e7c632ae9b6fa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.549182 10862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c7a511512184f53921cea081f82aa37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.588321 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_85b1c71bf0984d6bb91bbc04608af545"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:40.590253 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85b1c71bf0984d6bb91bbc04608af545 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:40.494166 11352 tablet_service.cc:1511] Processing CreateTablet for tablet 2692a931500443d298e25a2b362b22a1 (DEFAULT_TABLE table=default.Table_82ff6d419e664eeaa2ac71124ae83342 [id=295bede819e9407e8a7e6a861d1b36ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.582697 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 (DEFAULT_TABLE table=default.Table_c92dfb9ed1d342f197b158bef20fc0ac [id=96595f8bb6b84744bf7fc74f074a51ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.640515 10869 tablet_service.cc:1511] Processing CreateTablet for tablet bb7a8b6d06064addb02979a7feedf756 (DEFAULT_TABLE table=default.Table_001ce3fb9e1e480f915bdbd88c394244 [id=43255c746b5c4fcc91454fcad2c801a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.640974 12186 leader_election.cc:290] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:40.641462 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb7a8b6d06064addb02979a7feedf756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.663264 11352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2692a931500443d298e25a2b362b22a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.495683 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 91941616dfdd4df3aaa7c8460876d273 (DEFAULT_TABLE table=default.Table_ab740dbd782f44eb8f423baeede1de0f [id=92630f3e58394023a931a5e2c40e8591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.497116 11364 tablet_service.cc:1511] Processing CreateTablet for tablet c3bf43978dc94c2b9a7f3c18a4677c40 (DEFAULT_TABLE table=default.Table_e03575a4e6bb46afb0346fad51b99eb5 [id=97f7c7d3e84a4032887e35cf5ea63006]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.498739 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 403cb09c1786491ab0d40502038bf9c0 (DEFAULT_TABLE table=default.Table_63e5e407241d4eaea93f27d8285ab33a [id=9e066a2a246442b78bfc003520f4cfbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.534205 12115 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:53476 (request call id 125) took 1706 ms. Trace:
W20260809 01:45:40.536485 11283 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s user 0.009s sys 0.002s
I20260809 01:45:40.538973 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 3e03033672e947eab8cc143263281133 (DEFAULT_TABLE table=default.Table_79500ae2591d480f93bd129b5e4e5804 [id=24d2e675d969421fba4647380880f606]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.540179 11369 tablet_service.cc:1511] Processing CreateTablet for tablet a38bd344108649ed8e1687f82154000b (DEFAULT_TABLE table=default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.541272 11351 tablet_service.cc:1511] Processing CreateTablet for tablet 761370ec8b164fd184a2e091030f610b (DEFAULT_TABLE table=default.Table_4d3af03d62f44239afda0e4122e17b2d [id=3e8bf3dc52ab433b84eeafb2c2685e21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.584714 11365 tablet_service.cc:1511] Processing CreateTablet for tablet 0354e4c7bf354f22a9c3c389e0e8ec92 (DEFAULT_TABLE table=default.Table_f81e3edafcf34038a6081ef4cd6ab8f3 [id=e1db9d7b43aa44e6b9a6e3ca7098c95f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.585898 11360 tablet_service.cc:1511] Processing CreateTablet for tablet cd09bea5263344919ce26593fe958547 (DEFAULT_TABLE table=default.Table_6f6560d974c74f2dbfe00a79fc671fcb [id=cb35a16a1e1241abb696b413e3c1d8cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.672509 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.692781 11365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0354e4c7bf354f22a9c3c389e0e8ec92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.693447 11554 tablet_bootstrap.cc:654] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:40.694164 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e03033672e947eab8cc143263281133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.707926 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91941616dfdd4df3aaa7c8460876d273. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:40.709730 12173 log.cc:927] Time spent T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8: Append to log took a long time: real 0.348s user 0.001s sys 0.000s
I20260809 01:45:40.721967 10867 tablet_service.cc:1511] Processing CreateTablet for tablet afb6c244108a4ff3a2df0585e5280294 (DEFAULT_TABLE table=default.Table_f84f34b0360549f39d13b657f425307c [id=96a100a09e6d4375ad92065772eba726]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.723198 10867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afb6c244108a4ff3a2df0585e5280294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.736817 11123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9493892c1ff48e7a0558ff0fcb7a0f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.737893 10859 tablet_service.cc:1511] Processing CreateTablet for tablet 325e3d19b83545a197c2e09fc9643b5e (DEFAULT_TABLE table=default.Table_cf7bd2b9aec04c73b8ed72debcab795d [id=ccd0ea07da604f859047f2eca85fc566]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.742426 10859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 325e3d19b83545a197c2e09fc9643b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.742619 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91941616dfdd4df3aaa7c8460876d273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.743436 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2692a931500443d298e25a2b362b22a1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:40.694648 11283 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.785s user 0.009s sys 0.003s
I20260809 01:45:40.545276 11107 tablet_service.cc:1511] Processing CreateTablet for tablet 8323a7a40a1445f980c0adcf6a6d16c7 (DEFAULT_TABLE table=default.Table_78c91fd32f104ad785b008e345a24e04 [id=dc279b3b97e3489e803a3408d5d76395]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.755357 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3bf43978dc94c2b9a7f3c18a4677c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.705471 12186 raft_consensus.cc:493] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:40.772850 10858 tablet_service.cc:1511] Processing CreateTablet for tablet e93334cc766044e79d37f293bc8556fb (DEFAULT_TABLE table=default.Table_2b01d8bf86d44f96b64752f511b02f50 [id=7cede7d9f9bf44f997b90481537918dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.773749 10588 master_service.cc:960] ReplaceTablet: received request to replace tablet c9d839443474498386220c288ff1721b from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:40.775224 10856 tablet_service.cc:1511] Processing CreateTablet for tablet 8323a7a40a1445f980c0adcf6a6d16c7 (DEFAULT_TABLE table=default.Table_78c91fd32f104ad785b008e345a24e04 [id=dc279b3b97e3489e803a3408d5d76395]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.777442 10856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8323a7a40a1445f980c0adcf6a6d16c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.768702 10860 tablet_service.cc:1511] Processing CreateTablet for tablet 9fffa48ccf58476da79dccd9c008b67c (DEFAULT_TABLE table=default.Table_3227dde355ef4460a73ac13726bf7249 [id=3e51bbb9edf142ecabc9c92777fd80b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.781363 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 403cb09c1786491ab0d40502038bf9c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.783985 12194 raft_consensus.cc:493] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:40.784651 12194 raft_consensus.cc:515] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:40.767016 12186 raft_consensus.cc:515] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:40.756283 12180 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.563s user 0.074s sys 0.112s
I20260809 01:45:40.797726 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e93334cc766044e79d37f293bc8556fb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:40.809073 12180 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.692s user 0.075s sys 0.113s
I20260809 01:45:40.820581 12186 leader_election.cc:290] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:40.822130 10860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fffa48ccf58476da79dccd9c008b67c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.828281 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_749b9564932646a498e0622597593241"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:40.832543 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 (DEFAULT_TABLE table=default.Table_c92dfb9ed1d342f197b158bef20fc0ac [id=96595f8bb6b84744bf7fc74f074a51ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.741588 11951 ts_tablet_manager.cc:1950] T 34b103e2d3854eb4bf8e5cd649cbe129 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:40.832379 11364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3bf43978dc94c2b9a7f3c18a4677c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.830219 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84f1943c1a34e999fc7c12a5847cdb6" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:40.847927 11553 tablet_bootstrap.cc:654] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:40.830567 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 921040us
I20260809 01:45:40.852190 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38bd344108649ed8e1687f82154000b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.852918 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 403cb09c1786491ab0d40502038bf9c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.866436 12194 leader_election.cc:290] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:40.872198 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_749b9564932646a498e0622597593241 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:40.798138 11125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e03033672e947eab8cc143263281133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.885705 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:40.887207 10805 leader_election.cc:343] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:40.890604 11120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38bd344108649ed8e1687f82154000b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.891647 11119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0354e4c7bf354f22a9c3c389e0e8ec92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.909888 12194 raft_consensus.cc:493] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:40.910364 12194 raft_consensus.cc:515] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:40.918999 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 761370ec8b164fd184a2e091030f610b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.923993 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd09bea5263344919ce26593fe958547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:40.929872 12194 leader_election.cc:290] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:40.921116 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e4315b3c644846ac572883c51f23de" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:40.935626 11346 tablet_service.cc:1511] Processing CreateTablet for tablet bb7a8b6d06064addb02979a7feedf756 (DEFAULT_TABLE table=default.Table_001ce3fb9e1e480f915bdbd88c394244 [id=43255c746b5c4fcc91454fcad2c801a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.921473 11349 tablet_service.cc:1511] Processing CreateTablet for tablet 7c7a511512184f53921cea081f82aa37 (DEFAULT_TABLE table=default.Table_bc35fdc444ce468082e5037ebe63cf03 [id=e566d88eebc84e5284e7c632ae9b6fa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:40.949640 11347 tablet_service.cc:1511] Processing CreateTablet for tablet afb6c244108a4ff3a2df0585e5280294 (DEFAULT_TABLE table=default.Table_f84f34b0360549f39d13b657f425307c [id=96a100a09e6d4375ad92065772eba726]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:40.954625 10805 leader_election.cc:343] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:40.664770 12115 rpcz_store.cc:276] 0809 01:45:38.827409 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.827544 (+ 135us) service_pool.cc:224] Handling call
0809 01:45:40.534146 (+1706602us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:39.123148 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0809 01:45:39.123586 (+ 438us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 01:45:39.562068 (+438482us) alter_schema_op.cc:66] Acquired schema lock
0809 01:45:39.562136 (+ 68us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0809 01:45:39.562337 (+ 201us) alter_schema_op.cc:128] START. Timestamp: 1786239939562148
0809 01:45:39.562370 (+ 33us) op_driver.cc:348] REPLICATION: starting
0809 01:45:39.563344 (+ 974us) log.cc:844] Serialized 173 byte log entry
0809 01:45:39.798864 (+235520us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b5400080628. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0eeb412c21 00007f0eeb41188c 00007f0ee9542d9b 00007f0ee95450fa 00007f0ee9545088 00007f0ee9545008 00007f0ee9544fc3 00007f0ee95441dc 00007f0eeaed4a16 00007f0eeaecf1c8 00007f0ee90c26f0
0809 01:45:39.798980 (+ 116us) op_driver.cc:464] REPLICATION: finished
0809 01:45:39.846942 (+ 47962us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0809 01:45:39.849356 (+ 2414us) log.cc:844] Serialized 5 byte log entry
0809 01:45:40.236869 (+387513us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:40.530026 (+293157us) log.cc:844] Serialized 14 byte log entry
0809 01:45:40.534020 (+ 3994us) alter_schema_op.cc:73] Released schema lock
0809 01:45:40.534050 (+ 30us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":47751,"prepare.queue_time_us":205332,"prepare.run_cpu_time_us":21362,"prepare.run_wall_time_us":442002,"raft.queue_time_us":454,"raft.run_cpu_time_us":1065,"raft.run_wall_time_us":1060,"replication_time_us":176564,"spinlock_wait_cycles":98581248,"thread_start_us":166613,"threads_started":1}]]}
I20260809 01:45:40.991111 12115 tablet.cc:1724] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 01:45:40.989892 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36a542c773b7438ead16c5b3af224cf9" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:40.989810 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac8498dff04b4e93303d7a35e62134" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:40.999768 11349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c7a511512184f53921cea081f82aa37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.001814 11554 tablet_bootstrap.cc:492] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:41.002202 11554 ts_tablet_manager.cc:1403] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.761s user 0.012s sys 0.002s
I20260809 01:45:41.010326 11949 ts_tablet_manager.cc:1929] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:41.010704 11949 log.cc:1199] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/4754f2e2a35f4a6a91fa333e2cf564e4
I20260809 01:45:41.010816 10586 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 2aac67e7819f4a70b1050b558868f27f (table default.Table_7f2585a3c363483ea2b745cde05d5e18 [id=04d4b772004b44e781c255cf271fab34]). Expected version 3 got 0
W20260809 01:45:41.016353 10586 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.67:40463 (request call id 21) took 1771 ms (client timeout 1000 ms). Trace:
W20260809 01:45:41.016712 10586 rpcz_store.cc:269] 0809 01:45:39.245415 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.304497 (+ 59082us) service_pool.cc:224] Handling call
0809 01:45:39.570637 (+266140us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:41.016286 (+1445649us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":603027328}
I20260809 01:45:41.018288 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_e03575a4e6bb46afb0346fad51b99eb5" } new_table_name: "default.Table_022d458d8ac1457c8e363dfc40bd6880" modify_external_catalogs: true
W20260809 01:45:41.021423 10589 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.66:57751 (request call id 34) took 1419 ms (client timeout 1000 ms). Trace:
I20260809 01:45:41.019527 10588 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet c9d839443474498386220c288ff1721b
W20260809 01:45:41.025586 10589 rpcz_store.cc:269] 0809 01:45:39.602047 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.605619 (+ 3572us) service_pool.cc:224] Handling call
0809 01:45:41.021365 (+1415746us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:40.696749 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:40.697095 (+ 346us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:40.697120 (+ 25us) write_op.cc:435] Acquired schema lock
0809 01:45:40.697133 (+ 13us) tablet.cc:662] Decoding operations
0809 01:45:40.697373 (+ 240us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:40.697404 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:40.697423 (+ 19us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:40.697566 (+ 143us) tablet.cc:701] Row locks acquired
0809 01:45:40.697579 (+ 13us) write_op.cc:260] PREPARE: finished
0809 01:45:40.697690 (+ 111us) write_op.cc:270] Start()
0809 01:45:40.697766 (+ 76us) write_op.cc:276] Timestamp: P: 1786239940697609 usec, L: 0
0809 01:45:40.697780 (+ 14us) op_driver.cc:348] REPLICATION: starting
0809 01:45:40.698204 (+ 424us) log.cc:844] Serialized 527 byte log entry
0809 01:45:41.004038 (+305834us) op_driver.cc:464] REPLICATION: finished
0809 01:45:41.005636 (+ 1598us) write_op.cc:301] APPLY: starting
0809 01:45:41.005718 (+ 82us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:41.005866 (+ 148us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:41.005964 (+ 98us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:41.006163 (+ 199us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:41.006374 (+ 211us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:41.006436 (+ 62us) write_op.cc:312] APPLY: finished
0809 01:45:41.006621 (+ 185us) log.cc:844] Serialized 23 byte log entry
0809 01:45:41.006816 (+ 195us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:41.006924 (+ 108us) write_op.cc:454] Released schema lock
0809 01:45:41.006985 (+ 61us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:41.007450 (+ 465us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:41.007506 (+ 56us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":1320,"apply.run_cpu_time_us":1756,"apply.run_wall_time_us":1755,"num_ops":1,"prepare.queue_time_us":264884,"prepare.run_cpu_time_us":1648,"prepare.run_wall_time_us":2580,"replication_time_us":306112}]]}
I20260809 01:45:41.029304 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet c9d839443474498386220c288ff1721b with delete_type TABLET_DATA_DELETED (replaced by tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:41.029479 10588 catalog_manager.cc:6807] ReplaceTablet: tablet c9d839443474498386220c288ff1721b deleted and replaced by tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b
I20260809 01:45:41.029830 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_753471f1b2c1464c8ed9a6591a0ebd9e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:41.033053 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_753471f1b2c1464c8ed9a6591a0ebd9e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:41.035110 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb7a8b6d06064addb02979a7feedf756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.046895 11949 ts_tablet_manager.cc:1950] T 4754f2e2a35f4a6a91fa333e2cf564e4 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:41.047678 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 125) took 2261 ms. Trace:
W20260809 01:45:41.050094 10586 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c3bf43978dc94c2b9a7f3c18a4677c40 consensus state has no leader
I20260809 01:45:41.054198 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_3aab7a7876fe46caadaed8dde7387416"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:41.054354 11554 raft_consensus.cc:359] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:41.057005 11554 raft_consensus.cc:385] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:41.057292 11554 raft_consensus.cc:740] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:41.021281 10599 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_f2379f27ebbb4266ab36dafd7d80f79f" } new_table_name: "default.Table_53624396796f4684acbc3787e6e05f6e" modify_external_catalogs: true
I20260809 01:45:41.059333 12208 tablet.cc:1724] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260809 01:45:41.065315 10581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.125:42063, user_credentials={real_user=slave}} blocked reactor thread for 94358.8us
W20260809 01:45:41.068749 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3aab7a7876fe46caadaed8dde7387416 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:41.074174 11347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afb6c244108a4ff3a2df0585e5280294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.060688 11554 consensus_queue.cc:260] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:41.076315 11554 ts_tablet_manager.cc:1434] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.074s user 0.005s sys 0.003s
W20260809 01:45:41.073004 10805 leader_election.cc:343] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:41.087240 11553 tablet_bootstrap.cc:492] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:41.087738 11553 ts_tablet_manager.cc:1403] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.555s user 0.012s sys 0.004s
I20260809 01:45:41.090569 12194 raft_consensus.cc:493] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:41.091302 12194 raft_consensus.cc:515] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:41.095669 12194 leader_election.cc:290] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:41.052616 11345 tablet_service.cc:1511] Processing CreateTablet for tablet 9fffa48ccf58476da79dccd9c008b67c (DEFAULT_TABLE table=default.Table_3227dde355ef4460a73ac13726bf7249 [id=3e51bbb9edf142ecabc9c92777fd80b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:41.117846 10588 master_service.cc:960] ReplaceTablet: received request to replace tablet c528b45de0d14c19af5d6836b7a14f1c from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:41.123070 11553 raft_consensus.cc:359] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:41.123667 11553 raft_consensus.cc:385] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:41.123930 11553 raft_consensus.cc:740] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:41.124622 11553 consensus_queue.cc:260] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:40.942178 11551 raft_consensus.cc:359] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:41.127061 11551 raft_consensus.cc:385] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:41.127301 11551 raft_consensus.cc:740] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:41.129415 11553 ts_tablet_manager.cc:1434] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.041s user 0.007s sys 0.003s
I20260809 01:45:41.129703 12161 raft_consensus.cc:493] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:41.132613 10805 leader_election.cc:343] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:41.136868 11554 tablet_bootstrap.cc:492] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:41.149710 11342 tablet_service.cc:1511] Processing CreateTablet for tablet 325e3d19b83545a197c2e09fc9643b5e (DEFAULT_TABLE table=default.Table_cf7bd2b9aec04c73b8ed72debcab795d [id=ccd0ea07da604f859047f2eca85fc566]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:41.047971 11949 rpcz_store.cc:276] 0809 01:45:38.786031 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.786854 (+ 823us) service_pool.cc:224] Handling call
0809 01:45:40.449117 (+1662263us) ts_tablet_manager.cc:1151] Deleting tablet 4754f2e2a35f4a6a91fa333e2cf564e4
0809 01:45:40.449166 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:40.647281 (+198115us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.010212 (+362931us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.047623 (+ 37411us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1661661}
W20260809 01:45:41.154049 10599 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c528b45de0d14c19af5d6836b7a14f1c consensus state has no leader
I20260809 01:45:41.157328 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 4754f2e2a35f4a6a91fa333e2cf564e4 (table default.Table_d1d088df5e7d452ab5a8771a8d3679dd [id=bd9dd208af1e4bd596ae7e3a4b219faf]) successfully deleted
I20260809 01:45:41.160879 12208 tablet.cc:1724] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:41.179495 11553 tablet_bootstrap.cc:492] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:41.182587 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 8323a7a40a1445f980c0adcf6a6d16c7 (DEFAULT_TABLE table=default.Table_78c91fd32f104ad785b008e345a24e04 [id=dc279b3b97e3489e803a3408d5d76395]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:41.193238 11553 tablet_bootstrap.cc:654] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:41.194433 11345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fffa48ccf58476da79dccd9c008b67c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.194483 11949 tablet_replica.cc:333] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:41.195353 11949 raft_consensus.cc:2243] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:41.195888 11949 raft_consensus.cc:2272] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:41.204195 11949 ts_tablet_manager.cc:1916] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:41.165380 11551 consensus_queue.cc:260] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:41.220968 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_25d50c77e3f5425888afe07872b75188"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:41.223377 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25d50c77e3f5425888afe07872b75188 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:41.224030 11106 tablet_service.cc:1558] Processing DeleteTablet for tablet 737d7047a0c74a03b3203112987c0cb2 with delete_type TABLET_DATA_DELETED (replaced by tablet 325e3d19b83545a197c2e09fc9643b5e) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:41.228223 11105 tablet_service.cc:1558] Processing DeleteTablet for tablet c795f81d2bc248eeb39b64d6aaa0370c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:41.229040 11105 tablet_service.cc:1558] Processing DeleteTablet for tablet 12fb852b8ea8478cb160a265459865c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 7706838afc9f4dbebc3213b82c4da126) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:41.229600 11105 tablet_service.cc:1558] Processing DeleteTablet for tablet c9d839443474498386220c288ff1721b with delete_type TABLET_DATA_DELETED (replaced by tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:41.130247 12161 raft_consensus.cc:515] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:41.247797 12172 raft_consensus.cc:493] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:41.229343 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 101) took 3800 ms. Trace:
I20260809 01:45:41.255888 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_d32d2e398c0f4c548296bb53315f1c05" } new_table_name: "default.Table_f9be5b6bf90441a4b7e96ad423794b71" modify_external_catalogs: true
I20260809 01:45:41.256460 12204 consensus_queue.cc:1048] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260809 01:45:41.238176 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd09bea5263344919ce26593fe958547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.271057 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84f1943c1a34e999fc7c12a5847cdb6" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:41.273609 11117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 761370ec8b164fd184a2e091030f610b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.276856 11112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fffa48ccf58476da79dccd9c008b67c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.285969 11157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e4315b3c644846ac572883c51f23de" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:41.287540 11110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e93334cc766044e79d37f293bc8556fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.292658 11115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.296739 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c7a511512184f53921cea081f82aa37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.248406 12172 raft_consensus.cc:515] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:41.307004 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 57655.9us
I20260809 01:45:41.358835 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_39b1a27949b04026a729a622750b6b57" } new_table_name: "default.Table_33b1a3980a3f46b6bd99b4935671527c" modify_external_catalogs: true
I20260809 01:45:41.363088 11336 tablet_service.cc:1558] Processing DeleteTablet for tablet c9d839443474498386220c288ff1721b with delete_type TABLET_DATA_DELETED (replaced by tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b) from {username='slave'} at 127.0.0.1:53476
W20260809 01:45:41.374991 10579 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet c3bf43978dc94c2b9a7f3c18a4677c40 (table default.Table_022d458d8ac1457c8e363dfc40bd6880 [id=97f7c7d3e84a4032887e35cf5ea63006], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet c3bf43978dc94c2b9a7f3c18a4677c40 consensus state has no leader
I20260809 01:45:41.376243 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_a97d9950868e4ee0b869484bdde6827f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:41.335970 11341 tablet_service.cc:1511] Processing CreateTablet for tablet e93334cc766044e79d37f293bc8556fb (DEFAULT_TABLE table=default.Table_2b01d8bf86d44f96b64752f511b02f50 [id=7cede7d9f9bf44f997b90481537918dc]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:41.378067 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a97d9950868e4ee0b869484bdde6827f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:41.381958 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_84167a58d3634ac7b44df384d0b7288f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:41.383592 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_84167a58d3634ac7b44df384d0b7288f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:41.291513 11951 rpcz_store.cc:276] 0809 01:45:37.395723 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:37.395992 (+ 269us) service_pool.cc:224] Handling call
0809 01:45:39.556476 (+2160484us) ts_tablet_manager.cc:1151] Deleting tablet 34b103e2d3854eb4bf8e5cd649cbe129
0809 01:45:39.556518 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:39.852324 (+295806us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:40.491860 (+639536us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.195891 (+704031us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21027,"tablet-delete.queue_time_us":2159685}
I20260809 01:45:41.431633 12209 tablet.cc:1724] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260809 01:45:41.442669 10586 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 75e4315b3c644846ac572883c51f23de consensus state has no leader
I20260809 01:45:41.444115 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_86ddcc80418e437f8fb35426024f6d19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:41.450453 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86ddcc80418e437f8fb35426024f6d19 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:41.487219 12186 raft_consensus.cc:493] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:41.487475 10592 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet ead75d766d0346f6823074d90ba8054e (table default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233]). Expected version 2 got 0
I20260809 01:45:41.488260 10592 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 85d0a9d73d4841c393e7d2cef41fee89 (table default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0 [id=ac1fb532e0824edaa1a3abc07758fbbe]). Expected version 1 got 0
I20260809 01:45:41.487829 12186 raft_consensus.cc:515] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:41.489156 10592 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet ef8a334741d3419eaa377338e0eaa889 (table default.Table_52926e76c3ad4bbcbc425c32d4473a3c [id=51771c6c664942e4b1df1132b5641ffa]). Expected version 1 got 0
I20260809 01:45:41.492487 12186 leader_election.cc:290] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:41.507256 10592 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 2aac67e7819f4a70b1050b558868f27f (table default.Table_7f2585a3c363483ea2b745cde05d5e18 [id=04d4b772004b44e781c255cf271fab34]). Expected version 3 got 2
I20260809 01:45:41.509590 12208 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:45982 (request call id 149) took 1085 ms. Trace:
W20260809 01:45:41.522212 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f9a0fc14dd684ccaa19d51261b9c610e has no consensus state
I20260809 01:45:41.526912 10592 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_57ef64be310d4250b77b141860e944d5 [id=212edfae1bf849c3b46674ec191ee76d]). Expected version 2 got 0
I20260809 01:45:41.528057 10592 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet c16897857df24c6da5744a99d5f687c0 (table default.Table_6d4e0b3afc864450bdf1a8fb35f7dc75 [id=9b307bc98a124b07a2f46fafb20ce1f0]). Expected version 1 got 0
I20260809 01:45:41.563640 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36a542c773b7438ead16c5b3af224cf9" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:41.565567 12194 raft_consensus.cc:493] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:41.493223 12186 raft_consensus.cc:493] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:41.566219 12194 raft_consensus.cc:515] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:41.566465 12186 raft_consensus.cc:515] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:41.505425 11949 ts_tablet_manager.cc:1929] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:41.570104 11949 log.cc:1199] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/6d35f291c92a45899eed0f2f10a3a1d8
I20260809 01:45:41.570803 11949 ts_tablet_manager.cc:1950] T 6d35f291c92a45899eed0f2f10a3a1d8 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:41.571529 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 122) took 2881 ms. Trace:
I20260809 01:45:41.571790 11949 rpcz_store.cc:276] 0809 01:45:38.690003 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.799141 (+109138us) service_pool.cc:224] Handling call
0809 01:45:41.150866 (+2351725us) ts_tablet_manager.cc:1151] Deleting tablet 6d35f291c92a45899eed0f2f10a3a1d8
0809 01:45:41.194324 (+ 43458us) spinlock_profiling.cc:243] Waited 42.7 ms on lock 0x7b5800000f98. stack: 00007fe5ab55ecf3 00007fe5ab55ebf0 00007fe5ab10c574 0000000000516de4 0000000000516d79 00007fe5ab419981 00007fe5ab559521 00007fe5ab559b52 00007fe5b44c71d9 00007fe5b36df4e8 00007fe5b37a5aa9 00007fe5b37a60a3 00007fe5b37d70e0 00007fe5b37bd64d 00007fe5b37bd5f9 00007fe5b37bd589
0809 01:45:41.194367 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:41.336693 (+142326us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.505256 (+168563us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.571481 (+ 66225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98116864,"tablet-delete.queue_time_us":2351228}
I20260809 01:45:41.572844 11949 tablet_replica.cc:333] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:41.575589 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "201329c676ec4b6abbb72b22519bc41d" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:41.204821 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 322389us
I20260809 01:45:41.583052 11157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac8498dff04b4e93303d7a35e62134" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:41.556376 11111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 325e3d19b83545a197c2e09fc9643b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.601909 10589 master_service.cc:960] ReplaceTablet: received request to replace tablet 2aac67e7819f4a70b1050b558868f27f from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:41.603035 12186 leader_election.cc:290] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:41.605973 11127 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 125) took 1229 ms. Trace:
I20260809 01:45:41.564036 12208 rpcz_store.cc:276] 0809 01:45:40.424415 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.424780 (+ 365us) service_pool.cc:224] Handling call
0809 01:45:41.509442 (+1084662us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:40.427270 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0809 01:45:40.428022 (+ 752us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 01:45:40.428108 (+ 86us) alter_schema_op.cc:66] Acquired schema lock
0809 01:45:40.428166 (+ 58us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0809 01:45:40.428402 (+ 236us) alter_schema_op.cc:128] START. Timestamp: 1786239940428237
0809 01:45:40.428456 (+ 54us) op_driver.cc:348] REPLICATION: starting
0809 01:45:40.429836 (+ 1380us) log.cc:844] Serialized 173 byte log entry
0809 01:45:40.715061 (+285225us) spinlock_profiling.cc:243] Waited 929 us on lock 0x7b140014f170. stack: 00007fe5ab55ecf3 00007fe5ab55ebf0 00007fe5ab10c574 0000000000516de4 0000000000516d79 00007fe5b24dad71 00007fe5b24f4ae7 00007fe5b24f4ced 00007fe5b24cc588 00007fe5b24c795a 00007fe5b24c6da6 00007fe5b24c69c8 00007fe5b24d0411 00007fe5b24d03c9 00007fe5b24d0359 00007fe5b24d0321
0809 01:45:41.144196 (+429135us) op_driver.cc:464] REPLICATION: finished
0809 01:45:41.160777 (+ 16581us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0809 01:45:41.171525 (+ 10748us) log.cc:844] Serialized 5 byte log entry
0809 01:45:41.507216 (+335691us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.508599 (+ 1383us) log.cc:844] Serialized 14 byte log entry
0809 01:45:41.509207 (+ 608us) alter_schema_op.cc:73] Released schema lock
0809 01:45:41.509238 (+ 31us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":16388,"prepare.queue_time_us":297,"prepare.run_cpu_time_us":18722,"prepare.run_wall_time_us":193456,"raft.queue_time_us":339,"raft.run_cpu_time_us":773,"raft.run_wall_time_us":772,"replication_time_us":715622,"spinlock_wait_cycles":2137344,"thread_start_us":587109,"threads_started":3,"wal-append.queue_time_us":124591}]]}
I20260809 01:45:41.607787 12194 leader_election.cc:290] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:41.531261 11553 tablet_bootstrap.cc:492] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:41.608664 11553 ts_tablet_manager.cc:1403] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.429s user 0.020s sys 0.010s
W20260809 01:45:41.608886 11366 consensus_meta.cc:287] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.738s user 0.007s sys 0.000s
I20260809 01:45:41.610918 11553 raft_consensus.cc:359] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:41.611555 11553 raft_consensus.cc:385] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:41.611824 11553 raft_consensus.cc:740] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:41.612416 11553 consensus_queue.cc:260] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:41.606596 12222 tablet_replica.cc:333] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:41.614027 12172 leader_election.cc:290] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:41.617280 11551 ts_tablet_manager.cc:1434] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 1.452s user 0.018s sys 0.034s
I20260809 01:45:41.573470 11949 raft_consensus.cc:2243] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:41.621973 11391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d1ed215ba644abbae09e78016b2c21" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:41.621517 11949 raft_consensus.cc:2272] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:41.622996 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637f0d6261f041b5aeff5fcc709a3dfc" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:41.624365 11949 ts_tablet_manager.cc:1916] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:41.624377 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "201329c676ec4b6abbb72b22519bc41d" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:41.626269 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e93334cc766044e79d37f293bc8556fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.614085 11127 rpcz_store.cc:276] 0809 01:45:40.371280 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.372248 (+ 968us) service_pool.cc:224] Handling call
0809 01:45:40.743389 (+371141us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.754857 (+ 11468us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.071412 (+316555us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.593671 (+522259us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:41.600086 (+ 6415us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:41.529147 10592 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet d0091542071441e5bad0c2dca819f37f (table default.Table_fb9b15123c274d7db877eac3e7906edc [id=266a0465cb4544eba6f3e9df1eacf724]). Expected version 2 got 0
I20260809 01:45:41.633836 10598 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 60d356fcc48441ca80f2df29b533404f
I20260809 01:45:41.634335 12161 leader_election.cc:290] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:41.635339 11342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 325e3d19b83545a197c2e09fc9643b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.635365 10598 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 176) took 2444 ms. Trace:
I20260809 01:45:41.638007 12200 tablet.cc:1724] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:41.635653 10598 rpcz_store.cc:276] 0809 01:45:39.191129 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.191252 (+ 123us) service_pool.cc:224] Handling call
0809 01:45:39.192544 (+ 1292us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:39.193083 (+ 539us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:39.193216 (+ 133us) catalog_manager.cc:2850] Locking tablets
0809 01:45:40.340138 (+1146922us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:40.807936 (+467798us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:40.808281 (+ 345us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:41.633733 (+825452us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:41.635304 (+ 1571us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:40.686507 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:40.686869 (+ 362us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:40.686892 (+ 23us) write_op.cc:435] Acquired schema lock
0809 01:45:40.686903 (+ 11us) tablet.cc:662] Decoding operations
0809 01:45:40.687173 (+ 270us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:40.687208 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:40.687229 (+ 21us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:40.687489 (+ 260us) tablet.cc:701] Row locks acquired
0809 01:45:40.687501 (+ 12us) write_op.cc:260] PREPARE: finished
0809 01:45:40.687608 (+ 107us) write_op.cc:270] Start()
0809 01:45:40.687682 (+ 74us) write_op.cc:276] Timestamp: P: 1786239940687541 usec, L: 0
0809 01:45:40.687698 (+ 16us) op_driver.cc:348] REPLICATION: starting
0809 01:45:40.688247 (+ 549us) log.cc:844] Serialized 756 byte log entry
0809 01:45:40.792229 (+103982us) op_driver.cc:464] REPLICATION: finished
0809 01:45:40.805371 (+ 13142us) write_op.cc:301] APPLY: starting
0809 01:45:40.805435 (+ 64us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:40.805558 (+ 123us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:40.805648 (+ 90us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:40.805894 (+ 246us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:40.806039 (+ 145us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:40.806091 (+ 52us) write_op.cc:312] APPLY: finished
0809 01:45:40.806372 (+ 281us) log.cc:844] Serialized 29 byte log entry
0809 01:45:40.807500 (+ 1128us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:40.807592 (+ 92us) write_op.cc:454] Released schema lock
0809 01:45:40.807638 (+ 46us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:40.807905 (+ 267us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:40.807947 (+ 42us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":12945,"apply.run_cpu_time_us":1627,"apply.run_wall_time_us":2505,"num_ops":2,"prepare.queue_time_us":344592,"prepare.run_cpu_time_us":2049,"prepare.run_wall_time_us":10157,"raft.queue_time_us":76,"raft.run_cpu_time_us":706,"raft.run_wall_time_us":1027,"replication_time_us":104400}]]}
I20260809 01:45:41.641753 11951 tablet_replica.cc:333] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:41.642664 11951 raft_consensus.cc:2243] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:41.642724 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_7f59bdaa42d240d681f6b296de78b329"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:41.643217 11951 raft_consensus.cc:2272] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:41.644265 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f59bdaa42d240d681f6b296de78b329 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:41.645185 11951 ts_tablet_manager.cc:1916] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:41.646944 10595 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 995a37a9e6b14889b876da5d76c89b84
I20260809 01:45:41.646938 12160 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:53476 (request call id 134) took 1984 ms. Trace:
I20260809 01:45:41.648447 10595 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 183) took 2119 ms. Trace:
I20260809 01:45:41.648743 10595 rpcz_store.cc:276] 0809 01:45:39.529408 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.543375 (+ 13967us) service_pool.cc:224] Handling call
0809 01:45:39.568953 (+ 25578us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c86954d6 00007fa2c624db34 00007fa2c624dac2 00007fa2c624d9f1 00007fa2c624d96c 00007fa2c624cc22 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:39.571342 (+ 2389us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:39.598475 (+ 27133us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:39.598601 (+ 126us) catalog_manager.cc:2850] Locking tablets
0809 01:45:40.335961 (+737360us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:40.661253 (+325292us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:40.765786 (+104533us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:41.646847 (+881061us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:41.648383 (+ 1536us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:40.592251 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:40.592665 (+ 414us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:40.592704 (+ 39us) write_op.cc:435] Acquired schema lock
0809 01:45:40.592718 (+ 14us) tablet.cc:662] Decoding operations
0809 01:45:40.593019 (+ 301us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:40.593057 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:40.593081 (+ 24us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:40.593361 (+ 280us) tablet.cc:701] Row locks acquired
0809 01:45:40.593379 (+ 18us) write_op.cc:260] PREPARE: finished
0809 01:45:40.593493 (+ 114us) write_op.cc:270] Start()
0809 01:45:40.593616 (+ 123us) write_op.cc:276] Timestamp: P: 1786239940593429 usec, L: 0
0809 01:45:40.593631 (+ 15us) op_driver.cc:348] REPLICATION: starting
0809 01:45:40.594221 (+ 590us) log.cc:844] Serialized 756 byte log entry
0809 01:45:40.659835 (+ 65614us) op_driver.cc:464] REPLICATION: finished
0809 01:45:40.660141 (+ 306us) write_op.cc:301] APPLY: starting
0809 01:45:40.660186 (+ 45us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:40.660272 (+ 86us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:40.660361 (+ 89us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:40.660568 (+ 207us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:40.660677 (+ 109us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:40.660714 (+ 37us) write_op.cc:312] APPLY: finished
0809 01:45:40.660853 (+ 139us) log.cc:844] Serialized 29 byte log entry
0809 01:45:40.660964 (+ 111us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:40.661018 (+ 54us) write_op.cc:454] Released schema lock
0809 01:45:40.661065 (+ 47us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:40.686159 (+ 25094us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:40.686193 (+ 34us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":56625408,"child_traces":[["op",{"apply.queue_time_us":191,"apply.run_cpu_time_us":1099,"apply.run_wall_time_us":1099,"num_ops":2,"prepare.queue_time_us":254328,"prepare.run_cpu_time_us":2310,"prepare.run_wall_time_us":90819,"raft.queue_time_us":51562,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":12,"replication_time_us":66125}]]}
I20260809 01:45:41.654153 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8323a7a40a1445f980c0adcf6a6d16c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.661767 11354 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 141) took 1163 ms. Trace:
I20260809 01:45:41.624176 11123 tablet_service.cc:1511] Processing CreateTablet for tablet 33b297f9083744e2953d41cd7b1055df (DEFAULT_TABLE table=default.Table_e2eb9de1f32b4341ab1745468556e4b3 [id=86c378c67d6144db91aef2a5220bbacc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:41.662132 11354 rpcz_store.cc:276] 0809 01:45:40.498046 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.498151 (+ 105us) service_pool.cc:224] Handling call
0809 01:45:40.664230 (+166079us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.664319 (+ 89us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.176071 (+511752us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.316941 (+140870us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:41.661700 (+344759us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:41.624768 11129 tablet_service.cc:1511] Processing CreateTablet for tablet 63c1a610663a4a2fa58f6cdc5f8d86ce (DEFAULT_TABLE table=default.Table_6e83741f1d33450c91d06322e7b2ef98 [id=8a2084683a7740db80e2c52d3f4c9506]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:41.623600 11126 tablet_service.cc:1511] Processing CreateTablet for tablet 33a5ef5a3ff44f74ab19998ba9f19310 (DEFAULT_TABLE table=default.Table_d9c4bfa0bd764f46b7e9026317ed8e57 [id=6a406282dbbf41c18ce14b4260f145ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:41.656474 10598 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_98b0f2fc738b4afe8c25c3ff8886c862" } new_table_name: "default.Table_18e8960a1b8a457fb622abd881d541ed" modify_external_catalogs: true
I20260809 01:45:41.667754 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 977694496e014ef29f5d7d513e5219c0 (DEFAULT_TABLE table=default.Table_608518db25c1495388ad65d72dcbe817 [id=148da762500547349a600027071299ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:41.676766 11553 ts_tablet_manager.cc:1434] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.068s user 0.005s sys 0.000s
I20260809 01:45:41.650307 11105 tablet_service.cc:1511] Processing CreateTablet for tablet 50f45c02a6994c5aaaf9c0a499e15fed (DEFAULT_TABLE table=default.Table_eb6df7e248d44109898ab900441bda8b [id=7701d6a8eff047ddac4f8775385c57ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:41.637787 11107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8323a7a40a1445f980c0adcf6a6d16c7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:41.618542 11551 ts_tablet_manager.cc:1467] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7: Tablet startup took 1453ms
I20260809 01:45:41.620698 11157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12fb852b8ea8478cb160a265459865c4" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:41.688054 11157 raft_consensus.cc:3060] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:41.639667 12222 raft_consensus.cc:2243] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:41.699507 11553 tablet_bootstrap.cc:492] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:41.702111 12222 raft_consensus.cc:2272] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:41.707537 11353 consensus_meta.cc:287] T 3e03033672e947eab8cc143263281133 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.523s user 0.003s sys 0.006s
W20260809 01:45:41.708207 10591 catalog_manager.cc:5911] Failed to send AlterTable RPC for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_57ef64be310d4250b77b141860e944d5 [id=212edfae1bf849c3b46674ec191ee76d], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 7706838afc9f4dbebc3213b82c4da126 consensus state has no leader
W20260809 01:45:41.709131 10805 leader_election.cc:343] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:41.709924 11113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb7a8b6d06064addb02979a7feedf756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.711721 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac6eded297214b58a5d8129a60ca1120" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:41.723847 10805 leader_election.cc:343] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:41.730790 10591 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 28) took 2503 ms (client timeout 1000 ms). Trace:
I20260809 01:45:41.732344 10595 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_f9be5b6bf90441a4b7e96ad423794b71" } new_table_name: "default.Table_6097f876208540eca5173c69a4e0b6ba" modify_external_catalogs: true
I20260809 01:45:41.684290 11124 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 126) took 1312 ms. Trace:
I20260809 01:45:41.746188 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_20f4c0cc067f4302a50669bc5303985a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:41.747890 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_20f4c0cc067f4302a50669bc5303985a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:41.748534 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 34b103e2d3854eb4bf8e5cd649cbe129 (table default.Table_175d0b46158d49a88aac3833ba31533a [id=cf3f859364164b9085d525891266a825]) successfully deleted
I20260809 01:45:41.748852 11354 tablet_service.cc:1511] Processing CreateTablet for tablet fa12353c25cb4da680d478ce104bc6ba (DEFAULT_TABLE table=default.Table_a600cb6fbde0453695e637643d0f71e0 [id=998fda1d3be848938d8908f2cf4da207]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:41.680830 12180 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s user 0.019s sys 0.046s
I20260809 01:45:41.752007 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_f97a006f17844462abd4433315667e68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:41.753360 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f97a006f17844462abd4433315667e68 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:41.757028 10805 leader_election.cc:343] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:41.760103 10591 rpcz_store.cc:269] 0809 01:45:39.227384 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.227504 (+ 120us) service_pool.cc:224] Handling call
0809 01:45:41.730723 (+2503219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9472,"reported_tablets":31}
I20260809 01:45:41.647254 12160 rpcz_store.cc:276] 0809 01:45:39.662433 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.662564 (+ 131us) service_pool.cc:224] Handling call
0809 01:45:41.646878 (+1984314us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:39.666446 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0809 01:45:39.669730 (+ 3284us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 01:45:39.669763 (+ 33us) alter_schema_op.cc:66] Acquired schema lock
0809 01:45:39.669860 (+ 97us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0809 01:45:39.669989 (+ 129us) alter_schema_op.cc:128] START. Timestamp: 1786239939668082
0809 01:45:39.670013 (+ 24us) op_driver.cc:348] REPLICATION: starting
0809 01:45:39.670703 (+ 690us) log.cc:844] Serialized 173 byte log entry
0809 01:45:39.970111 (+299408us) op_driver.cc:464] REPLICATION: finished
0809 01:45:40.037811 (+ 67700us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0809 01:45:40.040807 (+ 2996us) log.cc:844] Serialized 5 byte log entry
0809 01:45:40.995888 (+955081us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.209601 (+213713us) log.cc:844] Serialized 14 byte log entry
0809 01:45:41.564745 (+355144us) alter_schema_op.cc:73] Released schema lock
0809 01:45:41.646757 (+ 82012us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":67440,"prepare.queue_time_us":365,"prepare.run_cpu_time_us":15729,"prepare.run_wall_time_us":218848,"raft.queue_time_us":116691,"raft.run_cpu_time_us":1646,"raft.run_wall_time_us":1646,"replication_time_us":299968,"thread_start_us":568553,"threads_started":3,"wal-append.queue_time_us":522589,"wal-append.run_cpu_time_us":2134,"wal-append.run_wall_time_us":1265781}]]}
I20260809 01:45:41.798285 10588 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet c528b45de0d14c19af5d6836b7a14f1c
I20260809 01:45:41.800413 10588 catalog_manager.cc:6807] ReplaceTablet: tablet c528b45de0d14c19af5d6836b7a14f1c deleted and replaced by tablet 31874b7e70a847a4885bd09da40a6902
W20260809 01:45:41.809175 10598 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7df6343423604a00a46f203f61e98e2f consensus state has no leader
I20260809 01:45:41.749929 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa12353c25cb4da680d478ce104bc6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.812389 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 2aac67e7819f4a70b1050b558868f27f
I20260809 01:45:41.760143 11366 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 139) took 1265 ms. Trace:
I20260809 01:45:41.813968 10589 catalog_manager.cc:6807] ReplaceTablet: tablet 2aac67e7819f4a70b1050b558868f27f deleted and replaced by tablet 9465f8bb39d64e1cab8646282cb65c75
I20260809 01:45:41.813910 11366 rpcz_store.cc:276] 0809 01:45:40.494991 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.495100 (+ 109us) service_pool.cc:224] Handling call
0809 01:45:40.663736 (+168636us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.663800 (+ 64us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:40.870396 (+206596us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.609296 (+738900us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:41.760090 (+150794us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:41.817031 10596 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet d0091542071441e5bad0c2dca819f37f (table default.Table_fb9b15123c274d7db877eac3e7906edc [id=266a0465cb4544eba6f3e9df1eacf724]). Expected version 2 got 0
I20260809 01:45:41.821034 10596 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 85d0a9d73d4841c393e7d2cef41fee89 (table default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0 [id=ac1fb532e0824edaa1a3abc07758fbbe]). Expected version 1 got 0
I20260809 01:45:41.773052 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389407e7ff464e0d8e84bbdbc4c6081e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:41.832085 11365 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 144) took 1293 ms. Trace:
I20260809 01:45:41.832585 11365 rpcz_store.cc:276] 0809 01:45:40.538854 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.540835 (+ 1981us) service_pool.cc:224] Handling call
0809 01:45:40.692103 (+151268us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.692240 (+ 137us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.237758 (+545518us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.756133 (+518375us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:41.831977 (+ 75844us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:41.835621 12211 raft_consensus.cc:493] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:41.836354 12211 raft_consensus.cc:515] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:41.839917 12211 leader_election.cc:290] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:41.840133 10596 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.67:40463 (request call id 22) took 1320 ms (client timeout 1000 ms). Trace:
W20260809 01:45:41.840452 10596 rpcz_store.cc:269] 0809 01:45:40.519983 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.767868 (+247885us) service_pool.cc:224] Handling call
0809 01:45:41.840057 (+1072189us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":7}
I20260809 01:45:41.762070 11114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afb6c244108a4ff3a2df0585e5280294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.844743 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63c1a610663a4a2fa58f6cdc5f8d86ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.742307 11124 rpcz_store.cc:276] 0809 01:45:40.371697 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.371829 (+ 132us) service_pool.cc:224] Handling call
0809 01:45:40.743221 (+371392us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.743325 (+ 104us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.208970 (+465645us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.605713 (+396743us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:41.684226 (+ 78513us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63232}
I20260809 01:45:41.846856 11124 tablet_service.cc:1558] Processing DeleteTablet for tablet 60d356fcc48441ca80f2df29b533404f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:39 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:41.849211 11123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33b297f9083744e2953d41cd7b1055df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:41.852546 10589 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_6d4e0b3afc864450bdf1a8fb35f7dc75" } modify_external_catalogs: true
I20260809 01:45:41.853338 10589 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_6d4e0b3afc864450bdf1a8fb35f7dc75" } modify_external_catalogs: true
W20260809 01:45:41.855505 10805 leader_election.cc:343] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:41.754601 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0091542071441e5bad0c2dca819f37f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:41.683616 11551 ts_tablet_manager.cc:1469] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7: Trace:
0809 01:45:33.673602 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:33.673730 (+ 128us) service_pool.cc:224] Handling call
0809 01:45:33.733237 (+ 59507us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:33.733319 (+ 82us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:33.754182 (+ 20863us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:33.797486 (+ 43304us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:33.857301 (+ 59815us) inbound_call.cc:177] Queueing success response
0809 01:45:39.488347 (+5631046us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0809 01:45:40.165148 (+676801us) ts_tablet_manager.cc:1435] Starting tablet replica
0809 01:45:40.554466 (+389318us) tablet_replica.cc:261] Starting instrumentation
0809 01:45:40.726086 (+171620us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":46267,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":185,"tablet-open.queue_time_us":5690532,"thread_start_us":333735,"threads_started":1}
I20260809 01:45:41.876971 11551 tablet_bootstrap.cc:492] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:41.878473 10596 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_03cfb94f08614dc5bcdaf9eb182adc1b" } modify_external_catalogs: true
I20260809 01:45:41.879473 10596 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_03cfb94f08614dc5bcdaf9eb182adc1b" } modify_external_catalogs: true
I20260809 01:45:41.757146 12222 ts_tablet_manager.cc:1916] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:41.752048 12180 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s user 0.020s sys 0.047s
W20260809 01:45:41.885835 10592 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 29) took 1499 ms (client timeout 1000 ms). Trace:
I20260809 01:45:41.888782 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 6d35f291c92a45899eed0f2f10a3a1d8 (table default.Table_14613c38aa0e4b258598254918b193bb [id=fa2e19a3042b41959165ca54e46ca690]) successfully deleted
W20260809 01:45:41.889576 10592 rpcz_store.cc:269] 0809 01:45:40.386161 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.418084 (+ 31923us) service_pool.cc:224] Handling call
0809 01:45:40.582344 (+164260us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:41.885745 (+1303401us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":205,"reported_tablets":33,"spinlock_wait_cycles":360109952}
W20260809 01:45:41.770911 11283 consensus_peers.cc:597] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0da8d25290814e66b99c2cc1d5f8b0cf. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:41.857636 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b51f999fab24b1ca7451ff543a124a4" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:41.862932 11353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 142) took 1363 ms. Trace:
I20260809 01:45:41.866138 11337 tablet_service.cc:1511] Processing CreateTablet for tablet 33a5ef5a3ff44f74ab19998ba9f19310 (DEFAULT_TABLE table=default.Table_d9c4bfa0bd764f46b7e9026317ed8e57 [id=6a406282dbbf41c18ce14b4260f145ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:41.868952 11336 tablet_service.cc:1511] Processing CreateTablet for tablet 63c1a610663a4a2fa58f6cdc5f8d86ce (DEFAULT_TABLE table=default.Table_6e83741f1d33450c91d06322e7b2ef98 [id=8a2084683a7740db80e2c52d3f4c9506]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:41.872834 11334 tablet_service.cc:1511] Processing CreateTablet for tablet 977694496e014ef29f5d7d513e5219c0 (DEFAULT_TABLE table=default.Table_608518db25c1495388ad65d72dcbe817 [id=148da762500547349a600027071299ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:41.875412 11333 tablet_service.cc:1558] Processing DeleteTablet for tablet 60d356fcc48441ca80f2df29b533404f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:39 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:41.878000 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 33b297f9083744e2953d41cd7b1055df (DEFAULT_TABLE table=default.Table_e2eb9de1f32b4341ab1745468556e4b3 [id=86c378c67d6144db91aef2a5220bbacc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:41.932979 10592 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet d0091542071441e5bad0c2dca819f37f (table default.Table_fb9b15123c274d7db877eac3e7906edc [id=266a0465cb4544eba6f3e9df1eacf724]). Expected version 2 got 0
I20260809 01:45:41.933806 10592 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 85d0a9d73d4841c393e7d2cef41fee89 (table default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0 [id=ac1fb532e0824edaa1a3abc07758fbbe]). Expected version 1 got 0
I20260809 01:45:41.953876 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_29e8714d4cdd4d308c5856c9b672239d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:41.955551 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_29e8714d4cdd4d308c5856c9b672239d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:41.966859 10595 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 75e4315b3c644846ac572883c51f23de consensus state has no leader
I20260809 01:45:41.984633 11949 ts_tablet_manager.cc:1929] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:41.985054 11949 log.cc:1199] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/36a542c773b7438ead16c5b3af224cf9
W20260809 01:45:42.005889 10804 leader_election.cc:343] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:42.006343 10804 leader_election.cc:304] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:42.011207 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_824915c2101a47bd9948a0d422cc8d16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:41.776590 12115 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:53476 (request call id 131) took 2365 ms. Trace:
I20260809 01:45:42.007081 12211 raft_consensus.cc:493] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:42.015103 12211 raft_consensus.cc:515] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:42.017005 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_824915c2101a47bd9948a0d422cc8d16 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:42.021309 10903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0091542071441e5bad0c2dca819f37f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:41.912117 11360 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 145) took 1372 ms. Trace:
I20260809 01:45:42.023919 11360 rpcz_store.cc:276] 0809 01:45:40.539246 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.541615 (+ 2369us) service_pool.cc:224] Handling call
0809 01:45:40.665810 (+124195us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.665903 (+ 93us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.599929 (+934026us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.911555 (+311626us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:41.912026 (+ 471us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:42.024976 11360 tablet_service.cc:1558] Processing DeleteTablet for tablet 995a37a9e6b14889b876da5d76c89b84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:39 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:42.005446 11369 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 143) took 1466 ms. Trace:
I20260809 01:45:42.027962 12211 leader_election.cc:290] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:42.045444 10804 leader_election.cc:343] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:42.045965 10804 leader_election.cc:304] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:41.941612 11554 tablet_bootstrap.cc:654] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:42.053944 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet c16897857df24c6da5744a99d5f687c0
I20260809 01:45:42.049404 10592 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_b92eb95256a74ee195ec93866ec1b673" } new_table_name: "default.Table_5f5f7b17133f4f4d9cfcac1aa0d270fa" modify_external_catalogs: true
I20260809 01:45:42.056167 10596 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet b2726cb09f734ed18bb9379f94f3f22a
I20260809 01:45:42.047166 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50f45c02a6994c5aaaf9c0a499e15fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.037284 11369 rpcz_store.cc:276] 0809 01:45:40.538550 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.539434 (+ 884us) service_pool.cc:224] Handling call
0809 01:45:40.665182 (+125748us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.665246 (+ 64us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.571583 (+906337us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.851640 (+280057us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:42.005388 (+153748us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:42.064922 11124 tablet_service.cc:1558] Processing DeleteTablet for tablet 995a37a9e6b14889b876da5d76c89b84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:39 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:42.065887 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 82d5a709cd86476594613e788a325094 (DEFAULT_TABLE table=default.Table_920e3c9eef6241a3bbe3da8898a42132 [id=9a9dabf9808b43cbaa2b1f966d9a94b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.024979 12231 raft_consensus.cc:493] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:42.069928 11949 ts_tablet_manager.cc:1950] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:42.070771 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 128) took 3159 ms. Trace:
I20260809 01:45:42.071348 11153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389407e7ff464e0d8e84bbdbc4c6081e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:42.071105 11949 rpcz_store.cc:276] 0809 01:45:38.911010 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.911149 (+ 139us) service_pool.cc:224] Handling call
0809 01:45:41.572707 (+2661558us) ts_tablet_manager.cc:1151] Deleting tablet 36a542c773b7438ead16c5b3af224cf9
0809 01:45:41.572742 (+ 35us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:41.913146 (+340404us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.984504 (+ 71358us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:42.070711 (+ 86207us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2660984}
I20260809 01:45:42.078171 11949 tablet_replica.cc:333] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:42.078917 11949 raft_consensus.cc:2243] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:42.079362 12246 tablet.cc:1724] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:42.085984 12180 raft_consensus.cc:493] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:42.086735 12180 raft_consensus.cc:515] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:42.091060 12180 leader_election.cc:290] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:42.096910 12226 raft_consensus.cc:2749] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:42.098927 12230 raft_consensus.cc:493] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:42.100466 11949 raft_consensus.cc:2272] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:42.099642 12230 raft_consensus.cc:515] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:42.101305 10804 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 55793.9us
W20260809 01:45:42.103170 10804 leader_election.cc:343] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:42.103629 10804 leader_election.cc:304] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:42.103672 12231 raft_consensus.cc:515] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:42.104432 10804 raft_consensus.cc:2686] T 36a542c773b7438ead16c5b3af224cf9 P b5117038a52a45b0aea3d30ca9cfcbe8: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 01:45:42.103288 12226 raft_consensus.cc:2749] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:42.111317 10804 leader_election.cc:343] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:42.111678 10804 leader_election.cc:304] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:42.115585 11553 tablet_bootstrap.cc:654] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:42.051483 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637f0d6261f041b5aeff5fcc709a3dfc" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:42.126544 12231 leader_election.cc:290] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:41.880986 11106 tablet_service.cc:1511] Processing CreateTablet for tablet fa12353c25cb4da680d478ce104bc6ba (DEFAULT_TABLE table=default.Table_a600cb6fbde0453695e637643d0f71e0 [id=998fda1d3be848938d8908f2cf4da207]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.067245 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d1ed215ba644abbae09e78016b2c21" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:42.135488 12230 leader_election.cc:290] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:42.063630 11120 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 128) took 1663 ms. Trace:
W20260809 01:45:42.139976 12251 log.cc:927] Time spent T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316: Append to log took a long time: real 0.130s user 0.001s sys 0.000s
I20260809 01:45:42.143435 10596 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet d0091542071441e5bad0c2dca819f37f (table default.Table_fb9b15123c274d7db877eac3e7906edc [id=266a0465cb4544eba6f3e9df1eacf724]). Expected version 2 got 0
I20260809 01:45:42.143716 12211 raft_consensus.cc:2749] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:41.979851 11289 consensus_peers.cc:597] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 737d7047a0c74a03b3203112987c0cb2. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:41.997728 11351 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 146) took 1458 ms. Trace:
I20260809 01:45:41.998821 11352 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 138) took 1505 ms. Trace:
I20260809 01:45:42.137123 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82d5a709cd86476594613e788a325094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.147676 11949 ts_tablet_manager.cc:1916] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:42.147938 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33a5ef5a3ff44f74ab19998ba9f19310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.149473 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 977694496e014ef29f5d7d513e5219c0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:42.155784 11063 leader_election.cc:343] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: da86b5e4a27544c59ae6bb387440d246
I20260809 01:45:42.159559 11553 tablet_bootstrap.cc:492] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:42.160102 11553 ts_tablet_manager.cc:1403] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.461s user 0.012s sys 0.002s
I20260809 01:45:42.163569 11553 raft_consensus.cc:359] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:42.164204 11553 raft_consensus.cc:385] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:42.164489 11553 raft_consensus.cc:740] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:42.165084 11553 consensus_queue.cc:260] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:42.176013 10598 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_608518db25c1495388ad65d72dcbe817" } new_table_name: "default.Table_36b7ad5cff5e46d99d519da60b890733" modify_external_catalogs: true
I20260809 01:45:42.188927 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_b6462a5c7160450f8605d2659d707573"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:42.190541 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b6462a5c7160450f8605d2659d707573 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:42.148973 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet 12fb852b8ea8478cb160a265459865c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 7706838afc9f4dbebc3213b82c4da126) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:42.174006 11365 tablet_service.cc:1558] Processing DeleteTablet for tablet c528b45de0d14c19af5d6836b7a14f1c with delete_type TABLET_DATA_DELETED (replaced by tablet 31874b7e70a847a4885bd09da40a6902) from {username='slave'} at 127.0.0.1:53476
W20260809 01:45:42.152817 10804 leader_election.cc:343] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:42.210695 10804 leader_election.cc:304] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:42.212253 10592 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 197b5043f6d348f89853c073b256daa4 consensus state has no leader
W20260809 01:45:42.214915 12259 log.cc:927] Time spent T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8: Append to log took a long time: real 0.054s user 0.000s sys 0.001s
W20260809 01:45:42.218608 10598 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 977694496e014ef29f5d7d513e5219c0 consensus state has no leader
I20260809 01:45:42.226972 11106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa12353c25cb4da680d478ce104bc6ba. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:42.238297 10804 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 85536.2us
I20260809 01:45:42.238217 12186 raft_consensus.cc:2749] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:42.239025 10804 leader_election.cc:343] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:42.239329 10804 leader_election.cc:304] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:42.239521 11553 ts_tablet_manager.cc:1434] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.079s user 0.005s sys 0.000s
I20260809 01:45:42.251868 12230 raft_consensus.cc:2749] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:42.252161 10804 leader_election.cc:343] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:42.252547 10804 leader_election.cc:304] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:42.253299 12186 raft_consensus.cc:2749] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:42.261294 11553 tablet_bootstrap.cc:492] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:42.261521 10598 master_service.cc:960] ReplaceTablet: received request to replace tablet 91941616dfdd4df3aaa7c8460876d273 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:42.268431 10589 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_5f5f7b17133f4f4d9cfcac1aa0d270fa" } modify_external_catalogs: true
I20260809 01:45:42.268882 10589 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_5f5f7b17133f4f4d9cfcac1aa0d270fa" } modify_external_catalogs: true
I20260809 01:45:42.272176 10597 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_6f6560d974c74f2dbfe00a79fc671fcb" } modify_external_catalogs: true
I20260809 01:45:42.272640 10597 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_6f6560d974c74f2dbfe00a79fc671fcb" } modify_external_catalogs: true
I20260809 01:45:42.279986 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_73c83efa21c2474984a83ffd88538ad2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:42.224237 11158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.10.8.65:50907 (request call id 91) took 1431 ms. Trace:
I20260809 01:45:42.141723 11120 rpcz_store.cc:276] 0809 01:45:40.396019 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.397706 (+ 1687us) service_pool.cc:224] Handling call
0809 01:45:40.781015 (+383309us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.793773 (+ 12758us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.640418 (+846645us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.881254 (+240836us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:42.059096 (+177842us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260809 01:45:42.281802 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_73c83efa21c2474984a83ffd88538ad2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:42.282371 11120 tablet_service.cc:1558] Processing DeleteTablet for tablet c528b45de0d14c19af5d6836b7a14f1c with delete_type TABLET_DATA_DELETED (replaced by tablet 31874b7e70a847a4885bd09da40a6902) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:42.283089 11120 tablet_service.cc:1558] Processing DeleteTablet for tablet 2aac67e7819f4a70b1050b558868f27f with delete_type TABLET_DATA_DELETED (replaced by tablet 9465f8bb39d64e1cab8646282cb65c75) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:42.286378 11120 tablet_service.cc:1511] Processing CreateTablet for tablet 81da8d08131d4a3ba636ecfc7a7d2951 (DEFAULT_TABLE table=default.Table_ce7493c5349e41d890087f1336689d4a [id=393bf4cab7eb48859853968aeae87b42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.287456 11120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81da8d08131d4a3ba636ecfc7a7d2951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.292783 11553 tablet_bootstrap.cc:654] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:42.280889 11158 rpcz_store.cc:276] 0809 01:45:40.809750 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.062255 (+252505us) service_pool.cc:224] Handling call
0809 01:45:41.231588 (+169333us) raft_consensus.cc:1471] Updating replica for 1 ops
0809 01:45:41.240896 (+ 9308us) raft_consensus.cc:1534] Early marking committed up to index 0
0809 01:45:41.258360 (+ 17464us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0809 01:45:41.307620 (+ 49260us) alter_schema_op.cc:128] START. Timestamp: 1786239940428237
0809 01:45:41.761327 (+453707us) log.cc:844] Serialized 173 byte log entry
0809 01:45:42.128832 (+367505us) raft_consensus.cc:1645] Marking committed up to 1
0809 01:45:42.136793 (+ 7961us) raft_consensus.cc:1697] Filling consensus response to leader.
0809 01:45:42.160416 (+ 23623us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0809 01:45:42.162594 (+ 2178us) raft_consensus.cc:1685] finished
0809 01:45:42.165396 (+ 2802us) raft_consensus.cc:1691] UpdateReplicas() finished
0809 01:45:42.217208 (+ 51812us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:41.696197 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0809 01:45:41.931484 (+235287us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 01:45:41.931570 (+ 86us) alter_schema_op.cc:66] Acquired schema lock
0809 01:45:42.025367 (+ 93797us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
Metrics: {"thread_start_us":353063,"threads_started":1,"wal-append.queue_time_us":311735,"child_traces":[["op",{"prepare.queue_time_us":137357,"prepare.run_cpu_time_us":2187,"prepare.run_wall_time_us":329261,"thread_start_us":104054,"threads_started":1}]]}
W20260809 01:45:42.316040 10804 leader_election.cc:343] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:42.316618 10804 leader_election.cc:304] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:42.317483 12186 raft_consensus.cc:2749] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:42.322889 10859 tablet_service.cc:1511] Processing CreateTablet for tablet 33a5ef5a3ff44f74ab19998ba9f19310 (DEFAULT_TABLE table=default.Table_d9c4bfa0bd764f46b7e9026317ed8e57 [id=6a406282dbbf41c18ce14b4260f145ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.325261 10860 tablet_service.cc:1511] Processing CreateTablet for tablet 33b297f9083744e2953d41cd7b1055df (DEFAULT_TABLE table=default.Table_e2eb9de1f32b4341ab1745468556e4b3 [id=86c378c67d6144db91aef2a5220bbacc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.327893 10858 tablet_service.cc:1511] Processing CreateTablet for tablet 63c1a610663a4a2fa58f6cdc5f8d86ce (DEFAULT_TABLE table=default.Table_6e83741f1d33450c91d06322e7b2ef98 [id=8a2084683a7740db80e2c52d3f4c9506]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.321231 10856 tablet_service.cc:1511] Processing CreateTablet for tablet fa12353c25cb4da680d478ce104bc6ba (DEFAULT_TABLE table=default.Table_a600cb6fbde0453695e637643d0f71e0 [id=998fda1d3be848938d8908f2cf4da207]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.331221 10867 tablet_service.cc:1511] Processing CreateTablet for tablet 977694496e014ef29f5d7d513e5219c0 (DEFAULT_TABLE table=default.Table_608518db25c1495388ad65d72dcbe817 [id=148da762500547349a600027071299ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.333721 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet 60d356fcc48441ca80f2df29b533404f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:39 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.335235 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet 995a37a9e6b14889b876da5d76c89b84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:39 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.339563 11553 tablet_bootstrap.cc:492] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:42.340034 11553 ts_tablet_manager.cc:1403] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.079s user 0.006s sys 0.022s
I20260809 01:45:42.343227 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 50f45c02a6994c5aaaf9c0a499e15fed (DEFAULT_TABLE table=default.Table_eb6df7e248d44109898ab900441bda8b [id=7701d6a8eff047ddac4f8775385c57ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.344007 11553 raft_consensus.cc:359] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:42.344904 11553 raft_consensus.cc:385] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:42.345264 11553 raft_consensus.cc:740] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:42.345355 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet 12fb852b8ea8478cb160a265459865c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 7706838afc9f4dbebc3213b82c4da126) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.346706 10867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 977694496e014ef29f5d7d513e5219c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.347532 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63c1a610663a4a2fa58f6cdc5f8d86ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.349066 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50f45c02a6994c5aaaf9c0a499e15fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.350170 10860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33b297f9083744e2953d41cd7b1055df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.359050 10856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa12353c25cb4da680d478ce104bc6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.362052 10859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33a5ef5a3ff44f74ab19998ba9f19310. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:42.174454 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 194794us
I20260809 01:45:42.175061 11351 rpcz_store.cc:276] 0809 01:45:40.539546 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.539928 (+ 382us) service_pool.cc:224] Handling call
0809 01:45:40.709080 (+169152us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.709162 (+ 82us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.577169 (+868007us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.917640 (+340471us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:41.997624 (+ 79984us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:42.369917 11949 ts_tablet_manager.cc:1929] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:42.370317 11949 log.cc:1199] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/0da8d25290814e66b99c2cc1d5f8b0cf
I20260809 01:45:42.371246 11949 ts_tablet_manager.cc:1950] T 0da8d25290814e66b99c2cc1d5f8b0cf P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:42.375070 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet c795f81d2bc248eeb39b64d6aaa0370c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.378012 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet 4754f2e2a35f4a6a91fa333e2cf564e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:45982
W20260809 01:45:42.380323 10581 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_57ef64be310d4250b77b141860e944d5 [id=212edfae1bf849c3b46674ec191ee76d], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7706838afc9f4dbebc3213b82c4da126 consensus state has no leader
I20260809 01:45:42.387084 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet 737d7047a0c74a03b3203112987c0cb2 with delete_type TABLET_DATA_DELETED (replaced by tablet 325e3d19b83545a197c2e09fc9643b5e) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.388078 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet ac6eded297214b58a5d8129a60ca1120 with delete_type TABLET_DATA_DELETED (replaced by tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.389056 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet c528b45de0d14c19af5d6836b7a14f1c with delete_type TABLET_DATA_DELETED (replaced by tablet 31874b7e70a847a4885bd09da40a6902) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.390185 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet 2aac67e7819f4a70b1050b558868f27f with delete_type TABLET_DATA_DELETED (replaced by tablet 9465f8bb39d64e1cab8646282cb65c75) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.392062 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet 5050a590fe0240ac8c9f9191ec2475a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.399004 10598 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 91941616dfdd4df3aaa7c8460876d273
I20260809 01:45:42.400668 10598 catalog_manager.cc:6807] ReplaceTablet: tablet 91941616dfdd4df3aaa7c8460876d273 deleted and replaced by tablet c11b0394533a485294ace9ee5e99bb19
I20260809 01:45:42.407186 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 129) took 3397 ms. Trace:
I20260809 01:45:42.407271 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_5c33dd5878b34a619bf65e590d9aca96"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:42.407596 11949 rpcz_store.cc:276] 0809 01:45:39.009989 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.010122 (+ 133us) service_pool.cc:224] Handling call
0809 01:45:42.078029 (+3067907us) ts_tablet_manager.cc:1151] Deleting tablet 0da8d25290814e66b99c2cc1d5f8b0cf
0809 01:45:42.078070 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:42.264496 (+186426us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:42.369770 (+105274us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:42.407107 (+ 37337us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1327,"tablet-delete.queue_time_us":3067273}
I20260809 01:45:42.409018 11949 tablet_replica.cc:333] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
W20260809 01:45:42.408872 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5c33dd5878b34a619bf65e590d9aca96 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:42.409767 11949 raft_consensus.cc:2243] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:42.410266 11949 raft_consensus.cc:2272] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:42.416065 10598 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_7f2585a3c363483ea2b745cde05d5e18" } new_table_name: "default.Table_68ce5e1e513c4cbda72e059398b158a9" modify_external_catalogs: true
I20260809 01:45:42.416468 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet cd09bea5263344919ce26593fe958547
I20260809 01:45:42.417065 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 197b5043f6d348f89853c073b256daa4
I20260809 01:45:42.422614 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_0ebfe415d8ef4946b8971a58797d97c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:42.048554 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 277815us
I20260809 01:45:42.185930 11369 tablet_service.cc:1511] Processing CreateTablet for tablet 50f45c02a6994c5aaaf9c0a499e15fed (DEFAULT_TABLE table=default.Table_eb6df7e248d44109898ab900441bda8b [id=7701d6a8eff047ddac4f8775385c57ae]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:42.424196 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ebfe415d8ef4946b8971a58797d97c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:42.362087 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 82d5a709cd86476594613e788a325094 (DEFAULT_TABLE table=default.Table_920e3c9eef6241a3bbe3da8898a42132 [id=9a9dabf9808b43cbaa2b1f966d9a94b0]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:42.397974 11349 consensus_meta.cc:287] T 7c7a511512184f53921cea081f82aa37 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.778s user 0.010s sys 0.000s
W20260809 01:45:42.399673 11346 consensus_meta.cc:287] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.808s user 0.007s sys 0.003s
W20260809 01:45:42.400764 11345 consensus_meta.cc:287] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.749s user 0.001s sys 0.009s
I20260809 01:45:42.431469 10586 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 85d0a9d73d4841c393e7d2cef41fee89 (table default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0 [id=ac1fb532e0824edaa1a3abc07758fbbe]). Expected version 1 got 0
W20260809 01:45:42.404479 11341 consensus_meta.cc:287] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.635s user 0.004s sys 0.005s
I20260809 01:45:42.186533 11352 rpcz_store.cc:276] 0809 01:45:40.493308 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.493446 (+ 138us) service_pool.cc:224] Handling call
0809 01:45:40.662805 (+169359us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.662891 (+ 86us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.596106 (+933215us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.998094 (+401988us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:41.998724 (+ 630us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:42.410944 11336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63c1a610663a4a2fa58f6cdc5f8d86ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.025081 11353 rpcz_store.cc:276] 0809 01:45:40.538216 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.538341 (+ 125us) service_pool.cc:224] Handling call
0809 01:45:40.693729 (+155388us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.693821 (+ 92us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.140274 (+446453us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.708258 (+567984us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:41.862870 (+154612us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:42.433044 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_b6681ecece924ef681c015f56e8cd4ae" } new_table_name: "default.Table_30d779db722c451baacdc0202550d5b6" modify_external_catalogs: true
I20260809 01:45:42.450314 10586 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet ef8a334741d3419eaa377338e0eaa889 (table default.Table_52926e76c3ad4bbcbc425c32d4473a3c [id=51771c6c664942e4b1df1132b5641ffa]). Expected version 1 got 0
I20260809 01:45:42.453893 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 36a542c773b7438ead16c5b3af224cf9 (table default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]) successfully deleted
I20260809 01:45:42.454712 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 0da8d25290814e66b99c2cc1d5f8b0cf (table default.Table_84e4a5312bbb43c9bd562d0f742537e2 [id=5cc4859efd27438a8570acaafe1e6e78]) successfully deleted
I20260809 01:45:42.455776 10586 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet d0091542071441e5bad0c2dca819f37f (table default.Table_fb9b15123c274d7db877eac3e7906edc [id=266a0465cb4544eba6f3e9df1eacf724]). Expected version 2 got 0
I20260809 01:45:42.421367 11333 tablet_service.cc:1558] Processing DeleteTablet for tablet 2aac67e7819f4a70b1050b558868f27f with delete_type TABLET_DATA_DELETED (replaced by tablet 9465f8bb39d64e1cab8646282cb65c75) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:42.422715 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 82d5a709cd86476594613e788a325094 (DEFAULT_TABLE table=default.Table_920e3c9eef6241a3bbe3da8898a42132 [id=9a9dabf9808b43cbaa2b1f966d9a94b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.462456 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82d5a709cd86476594613e788a325094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.424708 10866 tablet_service.cc:1511] Processing CreateTablet for tablet 81da8d08131d4a3ba636ecfc7a7d2951 (DEFAULT_TABLE table=default.Table_ce7493c5349e41d890087f1336689d4a [id=393bf4cab7eb48859853968aeae87b42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.426657 10862 tablet_service.cc:1511] Processing CreateTablet for tablet 1c73bed7a9774cbc8ac774e6514a981d (DEFAULT_TABLE table=default.Table_bf875ac0c8864c968aeb0483b1d7836f [id=ac5e8f07619c446aa7f6249afd741e14]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:42.463368 11289 leader_election.cc:343] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: ac6eded297214b58a5d8129a60ca1120
I20260809 01:45:42.468775 10586 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_57ef64be310d4250b77b141860e944d5 [id=212edfae1bf849c3b46674ec191ee76d]). Expected version 2 got 0
I20260809 01:45:42.469348 10586 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 7df6343423604a00a46f203f61e98e2f (table default.Table_18e8960a1b8a457fb622abd881d541ed [id=b7fa92e91f85479c82ba9d384c44d429]). Expected version 1 got 0
I20260809 01:45:42.469852 11365 tablet_service.cc:1511] Processing CreateTablet for tablet 1c73bed7a9774cbc8ac774e6514a981d (DEFAULT_TABLE table=default.Table_bf875ac0c8864c968aeb0483b1d7836f [id=ac5e8f07619c446aa7f6249afd741e14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.471704 11351 tablet_service.cc:1511] Processing CreateTablet for tablet 67b31998b65e4264b16c0d73dbcfc73b (DEFAULT_TABLE table=default.Table_2b3bf023a9304fa982b5bc6f1a0b701b [id=21fe6165acd14d51a03ef42828f9462d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:42.472718 11283 consensus_peers.cc:597] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:42.473286 10586 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 75e4315b3c644846ac572883c51f23de (table default.Table_6097f876208540eca5173c69a4e0b6ba [id=850f362a7f9a4531a3d7d6de29b695ba]). Expected version 3 got 0
I20260809 01:45:42.429136 10864 tablet_service.cc:1511] Processing CreateTablet for tablet 67b31998b65e4264b16c0d73dbcfc73b (DEFAULT_TABLE table=default.Table_2b3bf023a9304fa982b5bc6f1a0b701b [id=21fe6165acd14d51a03ef42828f9462d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.430987 10865 tablet_service.cc:1511] Processing CreateTablet for tablet c87361b29664464f9fd73be974e2d3a0 (DEFAULT_TABLE table=default.Table_7168034c2262465e932139aa19fd6fd4 [id=74e764815e2242a9be1ca850a0b6ff30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.431928 10877 tablet_service.cc:1558] Processing DeleteTablet for tablet c16897857df24c6da5744a99d5f687c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:41 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.432701 10874 tablet_service.cc:1558] Processing DeleteTablet for tablet b2726cb09f734ed18bb9379f94f3f22a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:41 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.474586 10586 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet cfb9afd3be194830ba99f5e20fdaca0a (table default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]). Expected version 1 got 0
I20260809 01:45:42.433454 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 19fe81e277cd4c798a07ceafa0b60566 (DEFAULT_TABLE table=default.Table_1998ec81597a4c4584dc4e4f90caa274 [id=cb347b72c1cf436095c6043c5b169641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.434886 10871 tablet_service.cc:1511] Processing CreateTablet for tablet 8baa4ae62edf46b5a386661a6826bbab (DEFAULT_TABLE table=default.Table_db89b9afbd84498e98b8c8fb48f0fd5d [id=b15bb9c641894a39a13c4409f4c51545]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.438766 10873 tablet_service.cc:1511] Processing CreateTablet for tablet f4d58189de244802b299e3b376319c1c (DEFAULT_TABLE table=default.Table_cf4d3615b483433d81b9cb61a945a25e [id=37ec9a8b5ec74d7fbd8bd053d283a573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.440305 10855 tablet_service.cc:1511] Processing CreateTablet for tablet a238ad3d46054daa9435e9076ba8311d (DEFAULT_TABLE table=default.Table_67456cecac004439bf596d7f3a46d602 [id=bd7272842da24f39ae1bc7b73cf5df5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.441004 11553 consensus_queue.cc:260] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:42.441459 10854 tablet_service.cc:1511] Processing CreateTablet for tablet a04881d1c5254053bef50dd3d9d475f2 (DEFAULT_TABLE table=default.Table_57234dd818414b7397d1ac621bdb06a7 [id=17cbc8bf77e142d08d03bd7a051e00ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.442437 10852 tablet_service.cc:1558] Processing DeleteTablet for tablet 91941616dfdd4df3aaa7c8460876d273 with delete_type TABLET_DATA_DELETED (replaced by tablet c11b0394533a485294ace9ee5e99bb19) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.443189 10847 tablet_service.cc:1558] Processing DeleteTablet for tablet 197b5043f6d348f89853c073b256daa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.451653 12226 raft_consensus.cc:493] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:42.452960 12230 raft_consensus.cc:493] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:42.455484 12186 raft_consensus.cc:493] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:42.474834 10866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81da8d08131d4a3ba636ecfc7a7d2951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.470290 10867 tablet_service.cc:1558] Processing DeleteTablet for tablet cd09bea5263344919ce26593fe958547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.479144 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 81da8d08131d4a3ba636ecfc7a7d2951 (DEFAULT_TABLE table=default.Table_ce7493c5349e41d890087f1336689d4a [id=393bf4cab7eb48859853968aeae87b42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.485143 10862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c73bed7a9774cbc8ac774e6514a981d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.489741 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet c16897857df24c6da5744a99d5f687c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:41 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:42.490537 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4d58189de244802b299e3b376319c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.490530 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet b2726cb09f734ed18bb9379f94f3f22a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:41 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:42.490301 12230 raft_consensus.cc:515] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:42.492274 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 19fe81e277cd4c798a07ceafa0b60566 (DEFAULT_TABLE table=default.Table_1998ec81597a4c4584dc4e4f90caa274 [id=cb347b72c1cf436095c6043c5b169641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.493248 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8baa4ae62edf46b5a386661a6826bbab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.495200 11333 tablet_service.cc:1511] Processing CreateTablet for tablet c87361b29664464f9fd73be974e2d3a0 (DEFAULT_TABLE table=default.Table_7168034c2262465e932139aa19fd6fd4 [id=74e764815e2242a9be1ca850a0b6ff30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.494997 12226 raft_consensus.cc:515] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:42.496551 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19fe81e277cd4c798a07ceafa0b60566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.497296 12186 raft_consensus.cc:515] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:42.502063 10864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67b31998b65e4264b16c0d73dbcfc73b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.504315 12186 leader_election.cc:290] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:42.476711 11283 leader_election.cc:343] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:42.520395 11949 ts_tablet_manager.cc:1916] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:42.521095 11352 tablet_service.cc:1511] Processing CreateTablet for tablet 8baa4ae62edf46b5a386661a6826bbab (DEFAULT_TABLE table=default.Table_db89b9afbd84498e98b8c8fb48f0fd5d [id=b15bb9c641894a39a13c4409f4c51545]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.521807 10589 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet d0091542071441e5bad0c2dca819f37f (table default.Table_fb9b15123c274d7db877eac3e7906edc [id=266a0465cb4544eba6f3e9df1eacf724]). Expected version 2 got 0
I20260809 01:45:42.522779 10589 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 85d0a9d73d4841c393e7d2cef41fee89 (table default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0 [id=ac1fb532e0824edaa1a3abc07758fbbe]). Expected version 1 got 0
I20260809 01:45:42.522727 11283 leader_election.cc:304] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:42.532737 12231 raft_consensus.cc:493] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:42.533483 12231 raft_consensus.cc:515] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:42.537678 12211 raft_consensus.cc:493] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:42.538425 12211 raft_consensus.cc:515] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:42.512732 10854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a04881d1c5254053bef50dd3d9d475f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.544188 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c87361b29664464f9fd73be974e2d3a0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:42.546962 11347 consensus_meta.cc:287] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.768s user 0.002s sys 0.007s
I20260809 01:45:42.500173 12230 leader_election.cc:290] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:42.511227 12226 leader_election.cc:290] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:42.554796 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a238ad3d46054daa9435e9076ba8311d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.564982 11951 ts_tablet_manager.cc:1929] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:42.565716 11553 ts_tablet_manager.cc:1434] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.225s user 0.002s sys 0.006s
I20260809 01:45:42.576398 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_52e499c473a24748b65fa3b389cf857a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:42.579068 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_52e499c473a24748b65fa3b389cf857a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:42.631315 10598 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9465f8bb39d64e1cab8646282cb65c75 has no consensus state
W20260809 01:45:42.638245 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8f0d79e03e66410487a72f1b47b327a0 consensus state has no leader
I20260809 01:45:42.557250 12211 leader_election.cc:290] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:42.555994 12231 leader_election.cc:290] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:42.671406 11551 tablet_bootstrap.cc:654] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:42.676915 10805 leader_election.cc:343] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:42.600059 11385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59de95452c0a4945beae5a9344537e0f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:42.600530 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef8a334741d3419eaa377338e0eaa889" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:42.600801 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc230f25baf94cbf80aa8e2771ec23ba" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:42.601578 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7df6343423604a00a46f203f61e98e2f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:42.602456 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb9afd3be194830ba99f5e20fdaca0a" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:42.680614 11951 log.cc:1199] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/5bb5e2ec60c24f8396daed992ca314ac
I20260809 01:45:42.682358 11951 ts_tablet_manager.cc:1950] T 5bb5e2ec60c24f8396daed992ca314ac P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:42.684593 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 100) took 5295 ms. Trace:
I20260809 01:45:42.606633 11334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 977694496e014ef29f5d7d513e5219c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.621835 11354 tablet_service.cc:1511] Processing CreateTablet for tablet f4d58189de244802b299e3b376319c1c (DEFAULT_TABLE table=default.Table_cf4d3615b483433d81b9cb61a945a25e [id=37ec9a8b5ec74d7fbd8bd053d283a573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.692097 11337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33a5ef5a3ff44f74ab19998ba9f19310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.698043 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4d58189de244802b299e3b376319c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.124010 12115 rpcz_store.cc:276] 0809 01:45:39.411118 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.412278 (+ 1160us) service_pool.cc:224] Handling call
0809 01:45:41.776484 (+2364206us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:39.792243 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0809 01:45:39.828080 (+ 35837us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 01:45:40.534545 (+706465us) alter_schema_op.cc:66] Acquired schema lock
0809 01:45:40.534595 (+ 50us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0809 01:45:40.534719 (+ 124us) alter_schema_op.cc:128] START. Timestamp: 1786239940534612
0809 01:45:40.534739 (+ 20us) op_driver.cc:348] REPLICATION: starting
0809 01:45:40.535197 (+ 458us) log.cc:844] Serialized 173 byte log entry
0809 01:45:40.921020 (+385823us) op_driver.cc:464] REPLICATION: finished
0809 01:45:40.991035 (+ 70015us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0809 01:45:41.003338 (+ 12303us) log.cc:844] Serialized 5 byte log entry
0809 01:45:41.774416 (+771078us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:41.775403 (+ 987us) log.cc:844] Serialized 14 byte log entry
0809 01:45:41.776276 (+ 873us) alter_schema_op.cc:73] Released schema lock
0809 01:45:41.776326 (+ 50us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":69879,"prepare.queue_time_us":377802,"prepare.run_cpu_time_us":17608,"prepare.run_wall_time_us":744433,"raft.queue_time_us":448,"raft.run_cpu_time_us":622,"raft.run_wall_time_us":621,"replication_time_us":386184,"thread_start_us":433787,"threads_started":1}]]}
I20260809 01:45:42.608721 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af35cf31a66474cb0575b2fb1547616" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:42.615443 11345 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 151) took 1634 ms. Trace:
I20260809 01:45:42.616708 11346 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 150) took 1763 ms. Trace:
I20260809 01:45:42.615846 11349 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 147) took 2075 ms. Trace:
I20260809 01:45:42.676003 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f86bba4bd94fbcacdc9c139e9de2d7" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:42.642342 11341 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 153) took 1580 ms. Trace:
I20260809 01:45:42.658528 11347 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 149) took 1821 ms. Trace:
I20260809 01:45:42.614091 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7706838afc9f4dbebc3213b82c4da126" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:42.713133 10595 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_085fd4536b524eb9aacb353d95ffa166" } modify_external_catalogs: true
I20260809 01:45:42.713153 10592 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_f97a006f17844462abd4433315667e68" } modify_external_catalogs: true
I20260809 01:45:42.713722 10595 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_085fd4536b524eb9aacb353d95ffa166" } modify_external_catalogs: true
I20260809 01:45:42.713722 10592 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_f97a006f17844462abd4433315667e68" } modify_external_catalogs: true
I20260809 01:45:42.718461 10586 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_ca43ce2300964cea8a0b69061d48fed0 [id=028331846d434fc1beff5b11e2129233] alter complete (version 2)
I20260809 01:45:42.684959 11951 rpcz_store.cc:276] 0809 01:45:37.393024 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:37.397608 (+ 4584us) service_pool.cc:224] Handling call
0809 01:45:41.582234 (+4184626us) ts_tablet_manager.cc:1151] Deleting tablet 5bb5e2ec60c24f8396daed992ca314ac
0809 01:45:41.641588 (+ 59354us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:42.364454 (+722866us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:42.564873 (+200419us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:42.684529 (+119656us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3995075}
W20260809 01:45:42.725116 11157 consensus_meta.cc:287] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 1.036s user 0.002s sys 0.021s
I20260809 01:45:42.725607 11157 raft_consensus.cc:2468] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:42.726545 11157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.10.8.67:33509 (request call id 62) took 1136 ms. Trace:
I20260809 01:45:42.726992 11157 rpcz_store.cc:276] 0809 01:45:41.590162 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.596279 (+ 6117us) service_pool.cc:224] Handling call
0809 01:45:42.726466 (+1130187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:42.730600 11126 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 145) took 1107 ms. Trace:
I20260809 01:45:42.731014 11126 rpcz_store.cc:276] 0809 01:45:41.622823 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.623022 (+ 199us) service_pool.cc:224] Handling call
0809 01:45:41.737475 (+114453us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.737577 (+ 102us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:42.668605 (+931028us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:42.711902 (+ 43297us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:42.730531 (+ 18629us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:42.754455 11126 tablet_service.cc:1511] Processing CreateTablet for tablet 1c73bed7a9774cbc8ac774e6514a981d (DEFAULT_TABLE table=default.Table_bf875ac0c8864c968aeb0483b1d7836f [id=ac5e8f07619c446aa7f6249afd741e14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.760500 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 36a542c773b7438ead16c5b3af224cf9 with delete_type TABLET_DATA_DELETED (replaced by tablet a38bd344108649ed8e1687f82154000b) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.762598 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50f45c02a6994c5aaaf9c0a499e15fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.763506 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82d5a709cd86476594613e788a325094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.755019 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 102) took 5287 ms. Trace:
I20260809 01:45:42.765561 11951 rpcz_store.cc:276] 0809 01:45:37.467838 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:37.469214 (+ 1376us) service_pool.cc:224] Handling call
0809 01:45:42.754276 (+5285062us) ts_tablet_manager.cc:1151] Deleting tablet 5050a590fe0240ac8c9f9191ec2475a1
0809 01:45:42.754563 (+ 287us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:42.754922 (+ 359us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5254907}
I20260809 01:45:42.767315 11553 tablet_bootstrap.cc:492] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:42.706346 11347 rpcz_store.cc:276] 0809 01:45:40.837064 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.853567 (+ 16503us) service_pool.cc:224] Handling call
0809 01:45:40.995379 (+141812us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f1984c 00007f0ee1f1940b 00007f0eeae5dfb9 00007f0eea09f640 00007f0eea184641 00007f0eea1618d3 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82 00007f0ee869cab1
0809 01:45:41.000445 (+ 5066us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.035506 (+ 35061us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.778652 (+743146us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:42.554063 (+775411us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:42.658461 (+104398us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38884224}
I20260809 01:45:42.774237 11347 tablet_service.cc:1511] Processing CreateTablet for tablet a238ad3d46054daa9435e9076ba8311d (DEFAULT_TABLE table=default.Table_67456cecac004439bf596d7f3a46d602 [id=bd7272842da24f39ae1bc7b73cf5df5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.777801 12200 tablet.cc:1724] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260809 01:45:42.779625 10805 leader_election.cc:343] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:42.780331 10805 leader_election.cc:343] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:42.780936 10805 leader_election.cc:343] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:42.781559 10805 leader_election.cc:343] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:42.782159 10805 leader_election.cc:343] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:42.793291 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_afe1f3910a684d509d8eb8112b74cb30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:42.793987 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_0ed496dc27464480aceaff109c421877"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:42.795318 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_afe1f3910a684d509d8eb8112b74cb30 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:42.782796 10805 leader_election.cc:343] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:42.798419 10805 leader_election.cc:343] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:42.799273 10805 leader_election.cc:343] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:42.795420 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ed496dc27464480aceaff109c421877 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:42.813717 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet 0da8d25290814e66b99c2cc1d5f8b0cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:42.844873 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_753471f1b2c1464c8ed9a6591a0ebd9e" } new_table_name: "default.Table_4f75e1e877e849fc99c4c673e332c032" modify_external_catalogs: true
I20260809 01:45:42.846321 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_e09e4f1ce6624e9b91ade11edfcb3e80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:42.847756 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e09e4f1ce6624e9b91ade11edfcb3e80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:42.847970 10586 master_service.cc:960] ReplaceTablet: received request to replace tablet 761370ec8b164fd184a2e091030f610b from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:42.849160 11949 ts_tablet_manager.cc:1929] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:42.825380 11553 tablet_bootstrap.cc:654] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:42.849495 11949 log.cc:1199] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/201329c676ec4b6abbb72b22519bc41d
I20260809 01:45:42.864048 10598 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_68ce5e1e513c4cbda72e059398b158a9" } new_table_name: "default.Table_72e975bfbe594052a24baee87785c71a" modify_external_catalogs: true
W20260809 01:45:42.756942 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 283416us
I20260809 01:45:42.866297 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67b31998b65e4264b16c0d73dbcfc73b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:42.877314 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 5050a590fe0240ac8c9f9191ec2475a1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5050a590fe0240ac8c9f9191ec2475a1 already in progress: creating tablet
I20260809 01:45:42.891299 11554 tablet_bootstrap.cc:492] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:42.891822 11554 ts_tablet_manager.cc:1403] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 1.755s user 0.016s sys 0.015s
I20260809 01:45:42.866601 11342 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 152) took 1851 ms. Trace:
I20260809 01:45:42.893751 11342 rpcz_store.cc:276] 0809 01:45:41.002813 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.054070 (+ 51257us) service_pool.cc:224] Handling call
0809 01:45:41.334974 (+280904us) spinlock_profiling.cc:243] Waited 33.9 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f193d2 00007f0eeae5dfb9 00007f0eea09f640 00007f0eea184641 00007f0eea1618d3 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82 00007f0ee869cab1
0809 01:45:41.335194 (+ 220us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.634883 (+299689us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:42.408983 (+774100us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:42.596311 (+187328us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:42.854206 (+257895us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77949184}
I20260809 01:45:42.910272 11949 ts_tablet_manager.cc:1950] T 201329c676ec4b6abbb72b22519bc41d P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:42.911254 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 130) took 3900 ms. Trace:
I20260809 01:45:42.911612 11949 rpcz_store.cc:276] 0809 01:45:39.011341 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.011449 (+ 108us) service_pool.cc:224] Handling call
0809 01:45:42.408857 (+3397408us) ts_tablet_manager.cc:1151] Deleting tablet 201329c676ec4b6abbb72b22519bc41d
0809 01:45:42.408908 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:42.793625 (+384717us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:42.849054 (+ 55429us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:42.911199 (+ 62145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36026,"tablet-delete.queue_time_us":3396863}
I20260809 01:45:42.912958 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 131) took 3694 ms. Trace:
I20260809 01:45:42.913074 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 201329c676ec4b6abbb72b22519bc41d (table default.Table_db89b9afbd84498e98b8c8fb48f0fd5d [id=b15bb9c641894a39a13c4409f4c51545]) successfully deleted
I20260809 01:45:42.862188 10873 tablet_service.cc:1511] Processing CreateTablet for tablet 34b334f049e24768ad2291326ae70121 (DEFAULT_TABLE table=default.Table_824915c2101a47bd9948a0d422cc8d16 [id=d8c7495a8995426c847fb77372c5e95a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.870699 10871 tablet_service.cc:1511] Processing CreateTablet for tablet acc9c8e0f97e41b48a919aa579ad85cd (DEFAULT_TABLE table=default.Table_20f4c0cc067f4302a50669bc5303985a [id=4bf0ff8c2175478ba0a5edab2b9ae7e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.874742 10860 tablet_service.cc:1511] Processing CreateTablet for tablet 3938bfd21b294fa6a25f0d8efc1ca29a (DEFAULT_TABLE table=default.Table_86ddcc80418e437f8fb35426024f6d19 [id=01721069a381465397d92855a7accc23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.899580 10856 tablet_service.cc:1511] Processing CreateTablet for tablet 00c7ec3928224059ab7f58d203224536 (DEFAULT_TABLE table=default.Table_753471f1b2c1464c8ed9a6591a0ebd9e [id=374e6a771a36443e9ecf07563041b587]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.914316 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc9c8e0f97e41b48a919aa579ad85cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.915194 10856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00c7ec3928224059ab7f58d203224536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.916498 10589 master_service.cc:960] ReplaceTablet: received request to replace tablet ef8a334741d3419eaa377338e0eaa889 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:42.922343 10860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3938bfd21b294fa6a25f0d8efc1ca29a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.705571 11345 rpcz_store.cc:276] 0809 01:45:40.981234 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.003337 (+ 22103us) service_pool.cc:224] Handling call
0809 01:45:41.183810 (+180473us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.193722 (+ 9912us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.651379 (+457657us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:42.497623 (+846244us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:42.615374 (+117751us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46725}
I20260809 01:45:42.928577 11345 tablet_service.cc:1558] Processing DeleteTablet for tablet 91941616dfdd4df3aaa7c8460876d273 with delete_type TABLET_DATA_DELETED (replaced by tablet c11b0394533a485294ace9ee5e99bb19) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:42.930190 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34b334f049e24768ad2291326ae70121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.930619 11345 tablet_service.cc:1511] Processing CreateTablet for tablet 34b334f049e24768ad2291326ae70121 (DEFAULT_TABLE table=default.Table_824915c2101a47bd9948a0d422cc8d16 [id=d8c7495a8995426c847fb77372c5e95a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.866829 12272 raft_consensus.cc:2749] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:42.899912 10859 tablet_service.cc:1511] Processing CreateTablet for tablet 7770aee1b7af415d995f63672fd05c01 (DEFAULT_TABLE table=default.Table_84167a58d3634ac7b44df384d0b7288f [id=d1ca00e3d6234799a396d40b3789e9f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.939103 10859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7770aee1b7af415d995f63672fd05c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.706185 11341 rpcz_store.cc:276] 0809 01:45:41.089877 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.141757 (+ 51880us) service_pool.cc:224] Handling call
0809 01:45:41.625618 (+483861us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.625734 (+ 116us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.768777 (+143043us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:42.504689 (+735912us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:42.642264 (+137575us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17204,"spinlock_wait_cycles":40320}
I20260809 01:45:42.943668 11341 tablet_service.cc:1558] Processing DeleteTablet for tablet 197b5043f6d348f89853c073b256daa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:42.944890 11341 tablet_service.cc:1558] Processing DeleteTablet for tablet cd09bea5263344919ce26593fe958547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:42.947409 12271 raft_consensus.cc:493] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:42.947983 10588 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_a58e460fc9644019b0d51c55b3673fea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:42.947975 12271 raft_consensus.cc:515] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:42.949774 10588 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a58e460fc9644019b0d51c55b3673fea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:42.880465 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da86b5e4a27544c59ae6bb387440d246" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:42.705699 11346 rpcz_store.cc:276] 0809 01:45:40.853239 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.911364 (+ 58125us) service_pool.cc:224] Handling call
0809 01:45:40.936600 (+ 25236us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.976573 (+ 39973us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.591825 (+615252us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:42.480092 (+888267us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:42.616654 (+136562us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38010}
I20260809 01:45:42.951645 12271 leader_election.cc:290] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:42.956951 12231 raft_consensus.cc:493] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:42.957664 12231 raft_consensus.cc:515] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:42.966481 12271 raft_consensus.cc:493] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:42.967700 12211 raft_consensus.cc:493] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:42.968348 12211 raft_consensus.cc:515] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:42.968735 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 60176118d61548ff96b001ab0dae7d21 (DEFAULT_TABLE table=default.Table_85b1c71bf0984d6bb91bbc04608af545 [id=b90c58e7b4a34ea2a22ec6e5aa4be490]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.970219 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60176118d61548ff96b001ab0dae7d21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.977607 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b (DEFAULT_TABLE table=default.Table_0bcdcc37d5f04f40ab4dd02b66baf4a6 [id=97ccdb35be6a437288222feb89f9e751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.978684 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.983038 11346 tablet_service.cc:1511] Processing CreateTablet for tablet acc9c8e0f97e41b48a919aa579ad85cd (DEFAULT_TABLE table=default.Table_20f4c0cc067f4302a50669bc5303985a [id=4bf0ff8c2175478ba0a5edab2b9ae7e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.984403 11365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c73bed7a9774cbc8ac774e6514a981d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.988760 12280 tablet.cc:1724] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:42.991510 11333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c87361b29664464f9fd73be974e2d3a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:42.997037 10866 tablet_service.cc:1511] Processing CreateTablet for tablet e360e79820d148f4a6e9fdeb2c19da54 (DEFAULT_TABLE table=default.Table_7f59bdaa42d240d681f6b296de78b329 [id=e800768eb8a2407c93290b889cc5f1a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.999209 10870 tablet_service.cc:1511] Processing CreateTablet for tablet f9a0fc14dd684ccaa19d51261b9c610e (DEFAULT_TABLE table=default.Table_33b1a3980a3f46b6bd99b4935671527c [id=2993a04ef2744f019c069da7ccc24532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.002046 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c528b45de0d14c19af5d6836b7a14f1c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:43.004081 10867 tablet_service.cc:1511] Processing CreateTablet for tablet bd3662c307ba4ff78b8f3165b23acb1a (DEFAULT_TABLE table=default.Table_f97a006f17844462abd4433315667e68 [id=0a517ec159a64c28aee43c00ca3f5359]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:43.007519 10805 leader_election.cc:343] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:42.967011 12271 raft_consensus.cc:515] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:43.020738 10595 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet b3b02bd8c93b43bf90ff0192a9541aae
W20260809 01:45:43.028038 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 00c7ec3928224059ab7f58d203224536 consensus state has no leader
I20260809 01:45:42.708973 11349 rpcz_store.cc:276] 0809 01:45:40.554208 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.843238 (+289030us) service_pool.cc:224] Handling call
0809 01:45:40.995601 (+152363us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.996611 (+ 1010us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.620058 (+623447us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:42.474649 (+854591us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:42.615770 (+141121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:43.020632 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19fe81e277cd4c798a07ceafa0b60566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.030563 10852 tablet_service.cc:1558] Processing DeleteTablet for tablet b3b02bd8c93b43bf90ff0192a9541aae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:43.032745 12231 leader_election.cc:290] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:43.037882 10586 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 761370ec8b164fd184a2e091030f610b
I20260809 01:45:43.039320 10586 catalog_manager.cc:6807] ReplaceTablet: tablet 761370ec8b164fd184a2e091030f610b deleted and replaced by tablet 21e28316ad9442e2bcffba726d13cfe9
I20260809 01:45:43.013434 10858 tablet_service.cc:1511] Processing CreateTablet for tablet 14b1d03085eb4ab69789fed6fab882ae (DEFAULT_TABLE table=default.Table_25d50c77e3f5425888afe07872b75188 [id=f5a337b5983945e7bc23ddd3818dfff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:42.998473 10866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e360e79820d148f4a6e9fdeb2c19da54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.042779 10852 tablet_service.cc:1558] Processing DeleteTablet for tablet 761370ec8b164fd184a2e091030f610b with delete_type TABLET_DATA_DELETED (replaced by tablet 21e28316ad9442e2bcffba726d13cfe9) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:43.050516 11553 tablet_bootstrap.cc:492] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:43.050978 11553 ts_tablet_manager.cc:1403] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.284s user 0.008s sys 0.005s
I20260809 01:45:43.055084 10592 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet bd3662c307ba4ff78b8f3165b23acb1a
I20260809 01:45:42.913254 11949 rpcz_store.cc:276] 0809 01:45:39.218292 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.221620 (+ 3328us) service_pool.cc:224] Handling call
0809 01:45:42.912736 (+3691116us) ts_tablet_manager.cc:1151] Deleting tablet b870347b07b04de5b833ab4fd3a2475f
0809 01:45:42.912773 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:42.912914 (+ 141us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3690597}
I20260809 01:45:42.896540 11342 tablet_service.cc:1511] Processing CreateTablet for tablet a04881d1c5254053bef50dd3d9d475f2 (DEFAULT_TABLE table=default.Table_57234dd818414b7397d1ac621bdb06a7 [id=17cbc8bf77e142d08d03bd7a051e00ad]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:43.071979 11289 consensus_peers.cc:597] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0da8d25290814e66b99c2cc1d5f8b0cf. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:43.074904 10867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd3662c307ba4ff78b8f3165b23acb1a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:43.075515 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet b870347b07b04de5b833ab4fd3a2475f because the tablet was not found. No further retry: Not found: Tablet not found: b870347b07b04de5b833ab4fd3a2475f
I20260809 01:45:43.076607 11349 tablet_service.cc:1511] Processing CreateTablet for tablet 7770aee1b7af415d995f63672fd05c01 (DEFAULT_TABLE table=default.Table_84167a58d3634ac7b44df384d0b7288f [id=d1ca00e3d6234799a396d40b3789e9f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.084440 12211 leader_election.cc:290] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:43.055681 11283 leader_election.cc:304] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:43.089035 10852 tablet_service.cc:1558] Processing DeleteTablet for tablet bd3662c307ba4ff78b8f3165b23acb1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:43.090298 12267 raft_consensus.cc:2749] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:45:43.090570 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_f10dd364f4ef4a7ab5e1b829e3412ca6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:43.092016 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 92389.4us
W20260809 01:45:43.092571 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f10dd364f4ef4a7ab5e1b829e3412ca6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:43.097611 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cef88ffe9e95437da81793795742def3" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:43.098299 10598 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9465f8bb39d64e1cab8646282cb65c75 consensus state has no leader
I20260809 01:45:43.100395 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet ef8a334741d3419eaa377338e0eaa889
I20260809 01:45:43.101693 10589 catalog_manager.cc:6807] ReplaceTablet: tablet ef8a334741d3419eaa377338e0eaa889 deleted and replaced by tablet 6f771c8051784da18a6de9fcf45e0125
W20260809 01:45:43.103231 10805 leader_election.cc:343] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:43.095942 11341 tablet_service.cc:1511] Processing CreateTablet for tablet 3938bfd21b294fa6a25f0d8efc1ca29a (DEFAULT_TABLE table=default.Table_86ddcc80418e437f8fb35426024f6d19 [id=01721069a381465397d92855a7accc23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.108498 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9a0fc14dd684ccaa19d51261b9c610e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.118563 11352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8baa4ae62edf46b5a386661a6826bbab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.121824 11364 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 140) took 2625 ms. Trace:
I20260809 01:45:43.122040 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81da8d08131d4a3ba636ecfc7a7d2951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.122115 11364 rpcz_store.cc:276] 0809 01:45:40.496361 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.496473 (+ 112us) service_pool.cc:224] Handling call
0809 01:45:40.706919 (+210446us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.706996 (+ 77us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.901537 (+1194541us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:42.126404 (+224867us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:42.414496 (+288092us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f19b52 00007f0eeae871d9 00007f0eea09f4e8 00007f0eea165783 00007f0eea16151f 00007f0eea161cdd 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82
0809 01:45:43.121774 (+707278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6434688}
I20260809 01:45:43.123099 10847 tablet_service.cc:1511] Processing CreateTablet for tablet 02e2f77a70794ff7811b5f09d885904b (DEFAULT_TABLE table=default.Table_749b9564932646a498e0622597593241 [id=61aaf5db03ef43b0b02226dea5f1879c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.123711 11364 tablet_service.cc:1511] Processing CreateTablet for tablet 14b1d03085eb4ab69789fed6fab882ae (DEFAULT_TABLE table=default.Table_25d50c77e3f5425888afe07872b75188 [id=f5a337b5983945e7bc23ddd3818dfff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.124672 11364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14b1d03085eb4ab69789fed6fab882ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.128497 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33b297f9083744e2953d41cd7b1055df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.128530 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 105) took 5206 ms. Trace:
I20260809 01:45:43.129056 11347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a238ad3d46054daa9435e9076ba8311d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.128846 11951 rpcz_store.cc:276] 0809 01:45:37.921763 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:37.921941 (+ 178us) service_pool.cc:224] Handling call
0809 01:45:42.767691 (+4845750us) ts_tablet_manager.cc:1151] Deleting tablet 7803a6aff02d42c1833f9b2cd7332b9f
0809 01:45:43.128234 (+360543us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:43.128480 (+ 246us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4845141}
I20260809 01:45:43.129240 10877 tablet_service.cc:1511] Processing CreateTablet for tablet 31874b7e70a847a4885bd09da40a6902 (DEFAULT_TABLE table=default.Table_53624396796f4684acbc3787e6e05f6e [id=18e87d9e1fa3442aaa489e3abcd4640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.131225 11342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a04881d1c5254053bef50dd3d9d475f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.132954 11345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34b334f049e24768ad2291326ae70121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.130446 10847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e2f77a70794ff7811b5f09d885904b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.135290 10874 tablet_service.cc:1558] Processing DeleteTablet for tablet ef8a334741d3419eaa377338e0eaa889 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f771c8051784da18a6de9fcf45e0125) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:43.134068 11951 tablet_replica.cc:333] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:43.144701 11951 raft_consensus.cc:2243] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:43.145659 11951 raft_consensus.cc:2272] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:43.147480 11951 ts_tablet_manager.cc:1916] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:43.109676 11554 raft_consensus.cc:359] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:43.156629 11554 raft_consensus.cc:385] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:43.156927 11554 raft_consensus.cc:740] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:43.156879 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 132) took 3919 ms. Trace:
I20260809 01:45:43.157316 11949 rpcz_store.cc:276] 0809 01:45:39.237876 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.238002 (+ 126us) service_pool.cc:224] Handling call
0809 01:45:43.073429 (+3835427us) ts_tablet_manager.cc:1151] Deleting tablet da86b5e4a27544c59ae6bb387440d246
0809 01:45:43.156583 (+ 83154us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:43.156812 (+ 229us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3834898}
I20260809 01:45:43.159417 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31874b7e70a847a4885bd09da40a6902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.159523 10874 tablet_service.cc:1511] Processing CreateTablet for tablet c704d148da8d406aa824253a51b850c8 (DEFAULT_TABLE table=default.Table_29e8714d4cdd4d308c5856c9b672239d [id=c064685015ef4b88ac9132f920977a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.160492 10874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c704d148da8d406aa824253a51b850c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.127775 12211 raft_consensus.cc:493] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:43.166121 12211 raft_consensus.cc:515] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:43.168020 12211 leader_election.cc:290] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:43.168478 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_b8741b4a79354d499ec83759f5887d31"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:43.170006 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8741b4a79354d499ec83759f5887d31 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:43.161741 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc9c8e0f97e41b48a919aa579ad85cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.185633 10589 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet d0091542071441e5bad0c2dca819f37f (table default.Table_fb9b15123c274d7db877eac3e7906edc [id=266a0465cb4544eba6f3e9df1eacf724]). Expected version 2 got 0
I20260809 01:45:43.170126 11369 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 164) took 1293 ms. Trace:
I20260809 01:45:43.189699 11369 rpcz_store.cc:276] 0809 01:45:41.876669 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.062070 (+185401us) service_pool.cc:224] Handling call
0809 01:45:42.761965 (+699895us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.762089 (+ 124us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:42.865706 (+103617us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.169682 (+303976us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.170060 (+ 378us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:43.191159 11369 tablet_service.cc:1511] Processing CreateTablet for tablet 00c7ec3928224059ab7f58d203224536 (DEFAULT_TABLE table=default.Table_753471f1b2c1464c8ed9a6591a0ebd9e [id=374e6a771a36443e9ecf07563041b587]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.192050 12231 raft_consensus.cc:493] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:43.184661 11554 consensus_queue.cc:260] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:43.192425 12231 raft_consensus.cc:515] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:43.196130 11554 ts_tablet_manager.cc:1434] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.304s user 0.005s sys 0.002s
I20260809 01:45:43.196971 11554 tablet_bootstrap.cc:492] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:43.198246 12267 raft_consensus.cc:493] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:43.198659 12267 raft_consensus.cc:515] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:43.204695 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 7803a6aff02d42c1833f9b2cd7332b9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7803a6aff02d42c1833f9b2cd7332b9f already in progress: creating tablet
I20260809 01:45:43.205892 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14b1d03085eb4ab69789fed6fab882ae. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:43.171603 11336 consensus_meta.cc:287] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.586s user 0.005s sys 0.006s
I20260809 01:45:43.043188 10864 tablet_service.cc:1511] Processing CreateTablet for tablet 100ad529186e4821b85ba28996933a8a (DEFAULT_TABLE table=default.Table_3aab7a7876fe46caadaed8dde7387416 [id=67b5f34f642646319bbfa3de7ed8895d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.210248 10864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 100ad529186e4821b85ba28996933a8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.222630 10855 tablet_service.cc:1511] Processing CreateTablet for tablet b3b02bd8c93b43bf90ff0192a9541aae (DEFAULT_TABLE table=default.Table_085fd4536b524eb9aacb353d95ffa166 [id=c0797cd687f2480087ce8379df004967]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.223701 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3b02bd8c93b43bf90ff0192a9541aae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.228760 10598 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_022d458d8ac1457c8e363dfc40bd6880" } modify_external_catalogs: true
I20260809 01:45:43.229382 10598 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_022d458d8ac1457c8e363dfc40bd6880" } modify_external_catalogs: true
I20260809 01:45:43.232295 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3938bfd21b294fa6a25f0d8efc1ca29a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.232906 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00c7ec3928224059ab7f58d203224536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.236441 11349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7770aee1b7af415d995f63672fd05c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.237820 10599 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_db89b9afbd84498e98b8c8fb48f0fd5d" } new_table_name: "default.Table_73716d58e8b348e2a0d3d697b9778fc0" modify_external_catalogs: true
I20260809 01:45:43.241029 12271 leader_election.cc:290] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:43.248968 12271 raft_consensus.cc:493] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:43.249650 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5050a590fe0240ac8c9f9191ec2475a1" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:43.249519 12271 raft_consensus.cc:515] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:43.249987 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16897857df24c6da5744a99d5f687c0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:43.252173 11336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 160) took 1384 ms. Trace:
I20260809 01:45:43.252583 11336 rpcz_store.cc:276] 0809 01:45:41.867445 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.867583 (+ 138us) service_pool.cc:224] Handling call
0809 01:45:42.134050 (+266467us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.145963 (+ 11913us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:42.585654 (+439691us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.232183 (+646529us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.252119 (+ 19936us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:43.253571 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 133) took 4014 ms. Trace:
I20260809 01:45:43.253881 11949 rpcz_store.cc:276] 0809 01:45:39.239062 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.239170 (+ 108us) service_pool.cc:224] Handling call
0809 01:45:43.158165 (+3918995us) ts_tablet_manager.cc:1151] Deleting tablet 5f8824953c794450ba274b160d891fcb
0809 01:45:43.158201 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:43.253495 (+ 95294us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3918577}
I20260809 01:45:43.254706 11336 tablet_service.cc:1511] Processing CreateTablet for tablet 100ad529186e4821b85ba28996933a8a (DEFAULT_TABLE table=default.Table_3aab7a7876fe46caadaed8dde7387416 [id=67b5f34f642646319bbfa3de7ed8895d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:43.254966 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 5f8824953c794450ba274b160d891fcb because the tablet was not found. No further retry: Not found: Tablet not found: 5f8824953c794450ba274b160d891fcb
I20260809 01:45:43.255191 11949 tablet_replica.cc:333] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:43.255796 11336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 100ad529186e4821b85ba28996933a8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.255894 11949 raft_consensus.cc:2243] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:43.256263 11949 raft_consensus.cc:2272] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:43.257697 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197b5043f6d348f89853c073b256daa4" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:43.260277 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6159d660d544b3e907a08372027c0e2" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:43.261538 10805 leader_election.cc:343] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:43.262326 10805 leader_election.cc:343] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:43.262502 12267 leader_election.cc:290] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
W20260809 01:45:43.262923 10805 leader_election.cc:343] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:43.263748 10805 leader_election.cc:343] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:43.268051 12200 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:53476 (request call id 155) took 2083 ms. Trace:
W20260809 01:45:43.279825 10581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.125:42063, user_credentials={real_user=slave}} blocked reactor thread for 47667.1us
I20260809 01:45:43.268400 12200 rpcz_store.cc:276] 0809 01:45:41.184794 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.244649 (+ 59855us) service_pool.cc:224] Handling call
0809 01:45:43.267996 (+2023347us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:41.593852 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0809 01:45:41.594279 (+ 427us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 01:45:41.594316 (+ 37us) alter_schema_op.cc:66] Acquired schema lock
0809 01:45:41.594401 (+ 85us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0809 01:45:41.594529 (+ 128us) alter_schema_op.cc:128] START. Timestamp: 1786239941594443
0809 01:45:41.594554 (+ 25us) op_driver.cc:348] REPLICATION: starting
0809 01:45:41.595167 (+ 613us) log.cc:844] Serialized 173 byte log entry
0809 01:45:42.761143 (+1165976us) op_driver.cc:464] REPLICATION: finished
0809 01:45:42.777679 (+ 16536us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0809 01:45:43.011302 (+233623us) log.cc:844] Serialized 5 byte log entry
0809 01:45:43.222959 (+211657us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.247925 (+ 24966us) log.cc:844] Serialized 14 byte log entry
0809 01:45:43.248869 (+ 944us) alter_schema_op.cc:73] Released schema lock
0809 01:45:43.267900 (+ 19031us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4712,"prepare.queue_time_us":346889,"prepare.run_cpu_time_us":2446,"prepare.run_wall_time_us":163962,"raft.queue_time_us":357,"raft.run_cpu_time_us":1633,"raft.run_wall_time_us":1633,"replication_time_us":1166427,"thread_start_us":494148,"threads_started":3,"wal-append.queue_time_us":187862,"wal-append.run_cpu_time_us":2829,"wal-append.run_wall_time_us":1030067}]]}
W20260809 01:45:43.314872 11064 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.650s user 0.001s sys 0.009s
W20260809 01:45:43.315490 11064 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.651s user 0.001s sys 0.009s
I20260809 01:45:43.320452 11283 leader_election.cc:304] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:43.321748 12267 raft_consensus.cc:2804] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:43.322679 12267 raft_consensus.cc:697] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:43.323539 12267 consensus_queue.cc:237] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:43.275774 10852 tablet_service.cc:1511] Processing CreateTablet for tablet 9465f8bb39d64e1cab8646282cb65c75 (DEFAULT_TABLE table=default.Table_68ce5e1e513c4cbda72e059398b158a9 [id=04d4b772004b44e781c255cf271fab34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.359268 10865 tablet_service.cc:1511] Processing CreateTablet for tablet 058dfc9b37f04fd8a72981b4290390b4 (DEFAULT_TABLE table=default.Table_a97d9950868e4ee0b869484bdde6827f [id=0cff66be49af4f69b1ddb65d69af11c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.286887 10586 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_63e5e407241d4eaea93f27d8285ab33a" } modify_external_catalogs: true
I20260809 01:45:43.370549 10586 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_63e5e407241d4eaea93f27d8285ab33a" } modify_external_catalogs: true
I20260809 01:45:43.301947 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_bf875ac0c8864c968aeb0483b1d7836f" } new_table_name: "default.Table_4657bf5b97b244a19b9e1f6c242a6552" modify_external_catalogs: true
I20260809 01:45:43.312323 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_0b1c340d21644c2da4547485ccbe16b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:43.374876 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0b1c340d21644c2da4547485ccbe16b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:43.395419 10581 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 00c7ec3928224059ab7f58d203224536 (table default.Table_4f75e1e877e849fc99c4c673e332c032 [id=374e6a771a36443e9ecf07563041b587], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 00c7ec3928224059ab7f58d203224536 consensus state has no leader
W20260809 01:45:43.434906 10854 consensus_meta.cc:287] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent flushing consensus metadata: real 0.574s user 0.010s sys 0.003s
I20260809 01:45:43.438043 12271 leader_election.cc:290] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:43.453575 10598 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet c3bf43978dc94c2b9a7f3c18a4677c40
W20260809 01:45:43.456604 10599 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8baa4ae62edf46b5a386661a6826bbab consensus state has no leader
W20260809 01:45:43.467619 11123 consensus_meta.cc:287] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 0.765s user 0.007s sys 0.000s
W20260809 01:45:43.471510 11119 consensus_meta.cc:287] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 1.419s user 0.007s sys 0.023s
I20260809 01:45:43.478551 11553 raft_consensus.cc:359] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:43.479260 11553 raft_consensus.cc:385] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:43.479614 11553 raft_consensus.cc:740] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:43.480425 11553 consensus_queue.cc:260] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:43.485194 10589 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0 [id=ac1fb532e0824edaa1a3abc07758fbbe] alter complete (version 1)
W20260809 01:45:43.489207 10581 consensus_peers.cc:597] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 -> Peer 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063): Couldn't send request to peer 71bc72deb1d2402cabd817dfe3b27e88. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260809 01:45:43.500612 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1c73bed7a9774cbc8ac774e6514a981d consensus state has no leader
W20260809 01:45:43.471211 11112 consensus_meta.cc:287] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 1.440s user 0.013s sys 0.014s
I20260809 01:45:43.512362 10586 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 403cb09c1786491ab0d40502038bf9c0
W20260809 01:45:43.565434 11354 consensus_meta.cc:287] T f4d58189de244802b299e3b376319c1c P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.793s user 0.007s sys 0.001s
I20260809 01:45:43.566454 11360 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 165) took 1689 ms. Trace:
I20260809 01:45:43.568696 11366 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 159) took 1701 ms. Trace:
I20260809 01:45:43.569096 11366 rpcz_store.cc:276] 0809 01:45:41.867124 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.877466 (+ 10342us) service_pool.cc:224] Handling call
0809 01:45:42.419656 (+542190us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f193d2 00007f0eeae5dfb9 00007f0eea09f640 00007f0eea184641 00007f0eea1618d3 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82 00007f0ee869cab1
0809 01:45:43.128098 (+708442us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.128213 (+ 115us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.204151 (+ 75938us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.564803 (+360652us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.568630 (+ 3827us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33244544}
W20260809 01:45:43.601599 11129 consensus_meta.cc:287] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 1.158s user 0.016s sys 0.001s
I20260809 01:45:43.604485 11123 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 146) took 1981 ms. Trace:
I20260809 01:45:43.604975 11123 rpcz_store.cc:276] 0809 01:45:41.623167 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.623671 (+ 504us) service_pool.cc:224] Handling call
0809 01:45:41.737707 (+114036us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.737799 (+ 92us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:42.662914 (+925115us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.468485 (+805571us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.604368 (+135883us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8448}
W20260809 01:45:43.607110 12246 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 1.390s user 0.007s sys 0.010s
I20260809 01:45:43.607474 11123 tablet_service.cc:1511] Processing CreateTablet for tablet 67b31998b65e4264b16c0d73dbcfc73b (DEFAULT_TABLE table=default.Table_2b3bf023a9304fa982b5bc6f1a0b701b [id=21fe6165acd14d51a03ef42828f9462d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.618237 11157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b51f999fab24b1ca7451ff543a124a4" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:43.619789 11117 consensus_meta.cc:287] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 1.002s user 0.011s sys 0.004s
W20260809 01:45:43.621238 11125 consensus_meta.cc:287] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 1.487s user 0.016s sys 0.006s
W20260809 01:45:43.621868 11127 consensus_meta.cc:287] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 1.180s user 0.014s sys 0.005s
I20260809 01:45:43.566866 11360 rpcz_store.cc:276] 0809 01:45:41.877054 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.136470 (+259416us) service_pool.cc:224] Handling call
0809 01:45:42.763065 (+626595us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.763148 (+ 83us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.216295 (+453147us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.336796 (+120501us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.566378 (+229582us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:43.634684 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 60176118d61548ff96b001ab0dae7d21 (DEFAULT_TABLE table=default.Table_85b1c71bf0984d6bb91bbc04608af545 [id=b90c58e7b4a34ea2a22ec6e5aa4be490]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.636528 11129 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 147) took 2013 ms. Trace:
I20260809 01:45:43.636914 11129 rpcz_store.cc:276] 0809 01:45:41.623485 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.624240 (+ 755us) service_pool.cc:224] Handling call
0809 01:45:41.737210 (+112970us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.737298 (+ 88us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:42.442932 (+705634us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.635644 (+1192712us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.636461 (+ 817us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10039}
I20260809 01:45:43.638952 11112 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 136) took 3131 ms. Trace:
I20260809 01:45:43.639427 11112 rpcz_store.cc:276] 0809 01:45:40.507583 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.508327 (+ 744us) service_pool.cc:224] Handling call
0809 01:45:41.235435 (+727108us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.235552 (+ 117us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:42.031189 (+795637us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.637994 (+1606805us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.638840 (+ 846us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5338}
I20260809 01:45:43.640606 11117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 131) took 3194 ms. Trace:
I20260809 01:45:43.483480 12231 leader_election.cc:290] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:43.495388 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 058dfc9b37f04fd8a72981b4290390b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.658838 10586 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_47a7750fcf1c4cf688f25aff5ded233d" } modify_external_catalogs: true
I20260809 01:45:43.659510 10586 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_47a7750fcf1c4cf688f25aff5ded233d" } modify_external_catalogs: true
I20260809 01:45:43.661640 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_57ef64be310d4250b77b141860e944d5" } new_table_name: "default.Table_996edb36352849a59f4ecb3388abaf8a" modify_external_catalogs: true
I20260809 01:45:43.665998 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_d917f500390043af96190d13685091ca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:43.667725 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d917f500390043af96190d13685091ca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:43.638875 11129 tablet_service.cc:1511] Processing CreateTablet for tablet c87361b29664464f9fd73be974e2d3a0 (DEFAULT_TABLE table=default.Table_7168034c2262465e932139aa19fd6fd4 [id=74e764815e2242a9be1ca850a0b6ff30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.644429 11127 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 148) took 1996 ms. Trace:
I20260809 01:45:43.654855 11119 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 129) took 3222 ms. Trace:
I20260809 01:45:43.673503 12279 tablet.cc:1724] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 01:45:43.674012 11119 rpcz_store.cc:276] 0809 01:45:40.433059 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.433280 (+ 221us) service_pool.cc:224] Handling call
0809 01:45:40.793786 (+360506us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.797873 (+ 4087us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:41.932696 (+1134823us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.653810 (+1721114us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.654785 (+ 975us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:43.643355 11125 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 127) took 3271 ms. Trace:
I20260809 01:45:43.682869 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_a36c740bfbe44b4891c5c03b3b684172"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:43.684412 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a36c740bfbe44b4891c5c03b3b684172 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:43.672956 11127 rpcz_store.cc:276] 0809 01:45:41.649200 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.667208 (+ 18008us) service_pool.cc:224] Handling call
0809 01:45:41.737118 (+ 69910us) spinlock_profiling.cc:243] Waited 9.72 ms on lock 0x7b5800000f98. stack: 00007fe4bc3e2cf3 00007fe4bc3e2bf0 00007fe4bbf90574 0000000000516de4 0000000000516d79 00007fe4bc29d981 00007fe4bc3dd84c 00007fe4bc3dd40b 00007fe4c5321fb9 00007fe4c4563640 00007fe4c4648641 00007fe4c46258d3 00007fe4c45d4210 00007fe4c2b60bf4 00007fe4c2b60b82 00007fe4c2b60ab1
0809 01:45:41.755713 (+ 18595us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.757949 (+ 2236us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:42.441490 (+683541us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.643685 (+1202195us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.644369 (+ 684us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21253,"spinlock_wait_cycles":22362368}
W20260809 01:45:43.668205 11118 consensus_meta.cc:287] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 1.443s user 0.007s sys 0.013s
I20260809 01:45:43.691817 11123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67b31998b65e4264b16c0d73dbcfc73b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.510694 11949 ts_tablet_manager.cc:1916] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:43.512547 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet c3bf43978dc94c2b9a7f3c18a4677c40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:43 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:43.613777 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet 403cb09c1786491ab0d40502038bf9c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:43 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:43.699080 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_20618fa8bbfc4193b40c62c5243d9645"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:43.700800 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_20618fa8bbfc4193b40c62c5243d9645 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:43.700974 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7706838afc9f4dbebc3213b82c4da126 consensus state has no leader
I20260809 01:45:43.704583 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60176118d61548ff96b001ab0dae7d21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.705214 10589 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_7168034c2262465e932139aa19fd6fd4" } new_table_name: "default.Table_0708cade7ac74a54aedeb8fe16b3b8df" modify_external_catalogs: true
I20260809 01:45:43.709692 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_b29e016ecc9343689732c208bfd2126e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:43.711856 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b29e016ecc9343689732c208bfd2126e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:43.669188 11154 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.757s user 0.011s sys 0.041s
I20260809 01:45:43.717762 10599 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_0b1c340d21644c2da4547485ccbe16b1" } modify_external_catalogs: true
I20260809 01:45:43.718386 10599 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_0b1c340d21644c2da4547485ccbe16b1" } modify_external_catalogs: true
I20260809 01:45:43.722425 12211 raft_consensus.cc:493] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:43.723096 12211 raft_consensus.cc:515] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:43.726369 11118 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 130) took 3284 ms. Trace:
I20260809 01:45:43.726812 11118 rpcz_store.cc:276] 0809 01:45:40.442674 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.443143 (+ 469us) service_pool.cc:224] Handling call
0809 01:45:40.809335 (+366192us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.077263 (+267928us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:42.138487 (+1061224us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.725658 (+1587171us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.726295 (+ 637us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:43.728137 11118 tablet_service.cc:1558] Processing DeleteTablet for tablet da86b5e4a27544c59ae6bb387440d246 with delete_type TABLET_DATA_DELETED (replaced by tablet 60d356fcc48441ca80f2df29b533404f) from {username='slave'} at 127.0.0.1:56566
W20260809 01:45:43.633471 10804 leader_election.cc:343] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:43.728835 10804 leader_election.cc:304] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:43.641192 11117 rpcz_store.cc:276] 0809 01:45:40.446590 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.446959 (+ 369us) service_pool.cc:224] Handling call
0809 01:45:41.071854 (+624895us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.166722 (+ 94868us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:42.617857 (+1451135us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.639721 (+1021864us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.640530 (+ 809us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19229}
W20260809 01:45:43.737022 11107 consensus_meta.cc:287] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 1.425s user 0.005s sys 0.013s
I20260809 01:45:43.676139 11119 tablet_service.cc:1558] Processing DeleteTablet for tablet b2726cb09f734ed18bb9379f94f3f22a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:41 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:43.742180 11107 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 139) took 3202 ms. Trace:
I20260809 01:45:43.641508 11112 tablet_service.cc:1558] Processing DeleteTablet for tablet c16897857df24c6da5744a99d5f687c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:41 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:43.739564 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 19fe81e277cd4c798a07ceafa0b60566 (DEFAULT_TABLE table=default.Table_1998ec81597a4c4584dc4e4f90caa274 [id=cb347b72c1cf436095c6043c5b169641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.731752 11553 ts_tablet_manager.cc:1434] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.680s user 0.000s sys 0.011s
I20260809 01:45:43.745913 12231 raft_consensus.cc:2749] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:43.746049 10804 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 112629us
W20260809 01:45:43.745736 11553 ts_tablet_manager.cc:1467] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8: Tablet startup took 694ms
W20260809 01:45:43.746891 11553 ts_tablet_manager.cc:1469] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8: Trace:
0809 01:45:38.690307 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.690418 (+ 111us) service_pool.cc:224] Handling call
0809 01:45:38.691563 (+ 1145us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:38.691633 (+ 70us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:38.700204 (+ 8571us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:38.717727 (+ 17523us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:38.718224 (+ 497us) inbound_call.cc:177] Queueing success response
0809 01:45:42.766995 (+4048771us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0809 01:45:43.051408 (+284413us) ts_tablet_manager.cc:1435] Starting tablet replica
0809 01:45:43.052839 (+ 1431us) tablet_replica.cc:261] Starting instrumentation
0809 01:45:43.235010 (+182171us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":233,"tablet-open.queue_time_us":4048893}
I20260809 01:45:43.751960 12211 leader_election.cc:290] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:43.733176 11951 ts_tablet_manager.cc:1929] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:45:43.752796 11951 log.cc:1199] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/5f8824953c794450ba274b160d891fcb
I20260809 01:45:43.753618 11951 ts_tablet_manager.cc:1950] T 5f8824953c794450ba274b160d891fcb P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:43.767376 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7706838afc9f4dbebc3213b82c4da126" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:43.769359 11157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef8a334741d3419eaa377338e0eaa889" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:43.777241 10847 consensus_meta.cc:287] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent flushing consensus metadata: real 0.589s user 0.006s sys 0.006s
I20260809 01:45:43.780002 11153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb9afd3be194830ba99f5e20fdaca0a" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:43.782298 11366 tablet_service.cc:1511] Processing CreateTablet for tablet e360e79820d148f4a6e9fdeb2c19da54 (DEFAULT_TABLE table=default.Table_7f59bdaa42d240d681f6b296de78b329 [id=e800768eb8a2407c93290b889cc5f1a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.790189 11152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59de95452c0a4945beae5a9344537e0f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:43.791642 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af35cf31a66474cb0575b2fb1547616" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:43.794493 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc230f25baf94cbf80aa8e2771ec23ba" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:43.796798 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c73bed7a9774cbc8ac774e6514a981d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.680137 11125 rpcz_store.cc:276] 0809 01:45:40.372271 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.372587 (+ 316us) service_pool.cc:224] Handling call
0809 01:45:40.754873 (+382286us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:40.781000 (+ 26127us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:42.118024 (+1337024us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.642037 (+1524013us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.643236 (+ 1199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20017}
I20260809 01:45:43.777269 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7df6343423604a00a46f203f61e98e2f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:43.827767 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c87361b29664464f9fd73be974e2d3a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.828604 11949 ts_tablet_manager.cc:1929] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:43.828989 11949 log.cc:1199] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/637f0d6261f041b5aeff5fcc709a3dfc
I20260809 01:45:43.831120 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f86bba4bd94fbcacdc9c139e9de2d7" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:43.832767 11119 tablet_service.cc:1511] Processing CreateTablet for tablet a04881d1c5254053bef50dd3d9d475f2 (DEFAULT_TABLE table=default.Table_57234dd818414b7397d1ac621bdb06a7 [id=17cbc8bf77e142d08d03bd7a051e00ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.847496 12266 raft_consensus.cc:493] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:43.847993 12266 raft_consensus.cc:515] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:43.850454 11112 tablet_service.cc:1511] Processing CreateTablet for tablet f4d58189de244802b299e3b376319c1c (DEFAULT_TABLE table=default.Table_cf4d3615b483433d81b9cb61a945a25e [id=37ec9a8b5ec74d7fbd8bd053d283a573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.853668 11949 ts_tablet_manager.cc:1950] T 637f0d6261f041b5aeff5fcc709a3dfc P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:43.743831 11107 rpcz_store.cc:276] 0809 01:45:40.541861 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.542277 (+ 416us) service_pool.cc:224] Handling call
0809 01:45:41.240600 (+698323us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.275114 (+ 34514us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:42.311865 (+1036751us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.741547 (+1429682us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.742115 (+ 568us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:43.857621 11107 tablet_service.cc:1511] Processing CreateTablet for tablet acc9c8e0f97e41b48a919aa579ad85cd (DEFAULT_TABLE table=default.Table_20f4c0cc067f4302a50669bc5303985a [id=4bf0ff8c2175478ba0a5edab2b9ae7e3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:43.698210 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 167543us
I20260809 01:45:43.861853 10852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9465f8bb39d64e1cab8646282cb65c75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.862836 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7803a6aff02d42c1833f9b2cd7332b9f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:43.879539 11333 consensus_meta.cc:287] T c87361b29664464f9fd73be974e2d3a0 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.762s user 0.006s sys 0.002s
I20260809 01:45:43.886144 10860 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45982 (request call id 196) took 1012 ms. Trace:
I20260809 01:45:43.886680 10860 rpcz_store.cc:276] 0809 01:45:42.873685 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.874076 (+ 391us) service_pool.cc:224] Handling call
0809 01:45:42.921617 (+ 47541us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.921837 (+ 220us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.080035 (+158198us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.447380 (+367345us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.886060 (+438680us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:43.888478 11554 tablet_bootstrap.cc:654] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:43.888644 10859 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45982 (request call id 197) took 1003 ms. Trace:
I20260809 01:45:43.889079 10859 rpcz_store.cc:276] 0809 01:45:42.885696 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.898729 (+ 13033us) service_pool.cc:224] Handling call
0809 01:45:42.938525 (+ 39796us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.938653 (+ 128us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.200738 (+262085us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.452947 (+252209us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.888565 (+435618us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:43.811439 11125 tablet_service.cc:1511] Processing CreateTablet for tablet a238ad3d46054daa9435e9076ba8311d (DEFAULT_TABLE table=default.Table_67456cecac004439bf596d7f3a46d602 [id=bd7272842da24f39ae1bc7b73cf5df5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.891984 10854 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45982 (request call id 188) took 1452 ms. Trace:
I20260809 01:45:43.892166 11117 tablet_service.cc:1558] Processing DeleteTablet for tablet 91941616dfdd4df3aaa7c8460876d273 with delete_type TABLET_DATA_DELETED (replaced by tablet c11b0394533a485294ace9ee5e99bb19) from {username='slave'} at 127.0.0.1:56566
W20260809 01:45:43.893031 11351 consensus_meta.cc:287] T 67b31998b65e4264b16c0d73dbcfc73b P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.559s user 0.008s sys 0.000s
I20260809 01:45:43.892489 10854 rpcz_store.cc:276] 0809 01:45:42.439659 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.440067 (+ 408us) service_pool.cc:224] Handling call
0809 01:45:42.505240 (+ 65173us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.505398 (+ 158us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:42.860372 (+354974us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.435501 (+575129us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.891904 (+456403us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:43.893369 11117 tablet_service.cc:1558] Processing DeleteTablet for tablet cd09bea5263344919ce26593fe958547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:43.894052 10588 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_12598d7dd6e4411793b992a49dbf090a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:43.895149 11553 tablet_bootstrap.cc:492] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:43.895329 11117 tablet_service.cc:1558] Processing DeleteTablet for tablet 197b5043f6d348f89853c073b256daa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:56566
W20260809 01:45:43.895728 10588 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_12598d7dd6e4411793b992a49dbf090a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:43.884615 11289 leader_election.cc:343] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: b870347b07b04de5b833ab4fd3a2475f
W20260809 01:45:43.900072 10805 leader_election.cc:343] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:43.904888 10856 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45982 (request call id 199) took 1006 ms. Trace:
I20260809 01:45:43.905820 10873 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45982 (request call id 194) took 1046 ms. Trace:
I20260809 01:45:43.906093 10873 rpcz_store.cc:276] 0809 01:45:42.859575 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.861626 (+ 2051us) service_pool.cc:224] Handling call
0809 01:45:42.929622 (+ 67996us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.929826 (+ 204us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.490448 (+560622us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.798291 (+307843us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.905758 (+107467us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:43.907291 10596 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_b8741b4a79354d499ec83759f5887d31" } new_table_name: "default.Table_c36eea2ca6e040ba888f487a8e539e04" modify_external_catalogs: true
I20260809 01:45:43.912386 10858 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45982 (request call id 198) took 1018 ms. Trace:
I20260809 01:45:43.914512 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19fe81e277cd4c798a07ceafa0b60566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.915478 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 134) took 4644 ms. Trace:
I20260809 01:45:43.915778 11949 rpcz_store.cc:276] 0809 01:45:39.271427 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.271566 (+ 139us) service_pool.cc:224] Handling call
0809 01:45:43.255046 (+3983480us) ts_tablet_manager.cc:1151] Deleting tablet 637f0d6261f041b5aeff5fcc709a3dfc
0809 01:45:43.255088 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:43.781562 (+526474us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.828495 (+ 46933us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.915429 (+ 86934us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3982926}
I20260809 01:45:43.905418 10856 rpcz_store.cc:276] 0809 01:45:42.898267 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.898375 (+ 108us) service_pool.cc:224] Handling call
0809 01:45:42.914367 (+ 15992us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.914528 (+ 161us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.485609 (+571081us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.794230 (+308621us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.904804 (+110574us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:43.917042 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 135) took 4641 ms. Trace:
I20260809 01:45:43.917340 11949 rpcz_store.cc:276] 0809 01:45:39.275156 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.275830 (+ 674us) service_pool.cc:224] Handling call
0809 01:45:43.916800 (+4640970us) ts_tablet_manager.cc:1151] Deleting tablet 135d3d65ea7846d3bc2cb59d44dbedf6
0809 01:45:43.916837 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:43.916992 (+ 155us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4640528}
I20260809 01:45:43.912952 10858 rpcz_store.cc:276] 0809 01:45:42.893811 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.005901 (+112090us) service_pool.cc:224] Handling call
0809 01:45:43.071925 (+ 66024us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.154707 (+ 82782us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.467094 (+312387us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.841234 (+374140us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.912299 (+ 71065us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:43.918479 11949 tablet_replica.cc:333] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
W20260809 01:45:43.919154 11346 consensus_meta.cc:287] T acc9c8e0f97e41b48a919aa579ad85cd P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.588s user 0.008s sys 0.000s
I20260809 01:45:43.919191 11949 raft_consensus.cc:2243] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260809 01:45:43.919490 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 135d3d65ea7846d3bc2cb59d44dbedf6 because the tablet was not found. No further retry: Not found: Tablet not found: 135d3d65ea7846d3bc2cb59d44dbedf6
I20260809 01:45:43.919641 11949 raft_consensus.cc:2272] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:43.919885 12266 leader_election.cc:290] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:43.921380 11949 ts_tablet_manager.cc:1916] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:43.922032 11160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c528b45de0d14c19af5d6836b7a14f1c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:43.885545 11353 consensus_meta.cc:287] T 19fe81e277cd4c798a07ceafa0b60566 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.576s user 0.007s sys 0.001s
I20260809 01:45:43.925412 11157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da86b5e4a27544c59ae6bb387440d246" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:43.926211 11157 raft_consensus.cc:2468] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:43.929250 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cef88ffe9e95437da81793795742def3" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:43.931798 11119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a04881d1c5254053bef50dd3d9d475f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.933576 11354 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 177) took 1517 ms. Trace:
I20260809 01:45:43.934034 11354 rpcz_store.cc:276] 0809 01:45:42.415943 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.618125 (+202182us) service_pool.cc:224] Handling call
0809 01:45:42.689814 (+ 71689us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.689906 (+ 92us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:42.771843 (+ 81937us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.599787 (+827944us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.933491 (+333704us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88081}
I20260809 01:45:43.935348 11553 tablet_bootstrap.cc:654] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:43.935812 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b (DEFAULT_TABLE table=default.Table_0bcdcc37d5f04f40ab4dd02b66baf4a6 [id=97ccdb35be6a437288222feb89f9e751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.936965 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.938041 11334 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 161) took 2067 ms. Trace:
W20260809 01:45:43.937765 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 397610us
W20260809 01:45:43.938963 11062 leader_election.cc:343] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:43.938426 11334 rpcz_store.cc:276] 0809 01:45:41.870381 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.870759 (+ 378us) service_pool.cc:224] Handling call
0809 01:45:42.187235 (+316476us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.605979 (+418744us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.186690 (+580711us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.693750 (+507060us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.937952 (+244202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5267}
I20260809 01:45:43.939747 11062 leader_election.cc:304] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:43.942435 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "995a37a9e6b14889b876da5d76c89b84" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:43.945495 10805 leader_election.cc:343] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:43.947106 11337 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 158) took 2200 ms. Trace:
I20260809 01:45:43.947608 11337 rpcz_store.cc:276] 0809 01:45:41.746641 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.863387 (+116746us) service_pool.cc:224] Handling call
0809 01:45:42.420649 (+557262us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f193d2 00007f0eeae5dfb9 00007f0eea09f640 00007f0eea184641 00007f0eea1618d3 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82 00007f0ee869cab1
0809 01:45:42.614262 (+193613us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.614378 (+ 116us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.176589 (+562211us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.778484 (+601895us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.947017 (+168533us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131513,"spinlock_wait_cycles":94649088}
I20260809 01:45:43.948654 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5050a590fe0240ac8c9f9191ec2475a1" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:43.950192 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16897857df24c6da5744a99d5f687c0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:43.950894 11347 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 178) took 1534 ms. Trace:
I20260809 01:45:43.951428 11347 rpcz_store.cc:276] 0809 01:45:42.416293 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.773455 (+357162us) service_pool.cc:224] Handling call
0809 01:45:43.128399 (+354944us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.128511 (+ 112us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.317653 (+189142us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.915883 (+598230us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.950794 (+ 34911us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125009}
I20260809 01:45:43.954056 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet b3b02bd8c93b43bf90ff0192a9541aae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:43.954838 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet 761370ec8b164fd184a2e091030f610b with delete_type TABLET_DATA_DELETED (replaced by tablet 21e28316ad9442e2bcffba726d13cfe9) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:43.955693 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet bd3662c307ba4ff78b8f3165b23acb1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:43.956337 11125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a238ad3d46054daa9435e9076ba8311d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.956243 10598 master_service.cc:960] ReplaceTablet: received request to replace tablet 8323a7a40a1445f980c0adcf6a6d16c7 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:43.956841 11369 tablet_service.cc:1511] Processing CreateTablet for tablet 02e2f77a70794ff7811b5f09d885904b (DEFAULT_TABLE table=default.Table_749b9564932646a498e0622597593241 [id=61aaf5db03ef43b0b02226dea5f1879c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.959851 11347 tablet_service.cc:1511] Processing CreateTablet for tablet bd3662c307ba4ff78b8f3165b23acb1a (DEFAULT_TABLE table=default.Table_f97a006f17844462abd4433315667e68 [id=0a517ec159a64c28aee43c00ca3f5359]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.961694 11112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4d58189de244802b299e3b376319c1c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:43.909551 11342 consensus_meta.cc:287] T a04881d1c5254053bef50dd3d9d475f2 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.538s user 0.004s sys 0.000s
I20260809 01:45:43.968327 11352 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 176) took 1553 ms. Trace:
I20260809 01:45:43.969962 11351 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 171) took 1841 ms. Trace:
I20260809 01:45:43.970444 11351 rpcz_store.cc:276] 0809 01:45:42.129136 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.471036 (+341900us) service_pool.cc:224] Handling call
0809 01:45:42.765844 (+294808us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.777176 (+ 11332us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.333829 (+556653us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.918660 (+584831us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.969904 (+ 51244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23575}
I20260809 01:45:43.972895 11351 tablet_service.cc:1558] Processing DeleteTablet for tablet ef8a334741d3419eaa377338e0eaa889 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f771c8051784da18a6de9fcf45e0125) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:43.941506 11334 tablet_service.cc:1511] Processing CreateTablet for tablet f9a0fc14dd684ccaa19d51261b9c610e (DEFAULT_TABLE table=default.Table_33b1a3980a3f46b6bd99b4935671527c [id=2993a04ef2744f019c069da7ccc24532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:43.976904 11334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9a0fc14dd684ccaa19d51261b9c610e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.981124 11107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc9c8e0f97e41b48a919aa579ad85cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.981040 11346 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 184) took 1071 ms. Trace:
I20260809 01:45:43.793840 11118 tablet_service.cc:1511] Processing CreateTablet for tablet 8baa4ae62edf46b5a386661a6826bbab (DEFAULT_TABLE table=default.Table_db89b9afbd84498e98b8c8fb48f0fd5d [id=b15bb9c641894a39a13c4409f4c51545]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:43.740736 11154 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.829s user 0.011s sys 0.042s
I20260809 01:45:43.984097 11154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.10.8.67:33509 (request call id 67) took 2328 ms. Trace:
I20260809 01:45:43.985571 11346 rpcz_store.cc:276] 0809 01:45:42.910049 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.981598 (+ 71549us) service_pool.cc:224] Handling call
0809 01:45:43.075481 (+ 93883us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f1984c 00007f0ee1f1940b 00007f0eeae5dfb9 00007f0eea09f640 00007f0eea184641 00007f0eea1618d3 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82 00007f0ee869cab1
0809 01:45:43.091908 (+ 16427us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f1989e 00007f0eeb413179 00007f0eea09f940 00007f0eea18465c 00007f0eea1618d3 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82 00007f0ee869cab1
0809 01:45:43.132231 (+ 40323us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.133225 (+ 994us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.331160 (+197935us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.919572 (+588412us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.980953 (+ 61381us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68281600}
I20260809 01:45:43.989446 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 7803a6aff02d42c1833f9b2cd7332b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:43.981987 11353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 175) took 1567 ms. Trace:
I20260809 01:45:43.989388 12211 raft_consensus.cc:493] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:43.990064 11353 rpcz_store.cc:276] 0809 01:45:42.414921 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.491106 (+ 76185us) service_pool.cc:224] Handling call
0809 01:45:42.889304 (+398198us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.996653 (+107349us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.223106 (+226453us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.925879 (+702773us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.981909 (+ 56030us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:43.990708 12211 raft_consensus.cc:515] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:43.982904 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e2f77a70794ff7811b5f09d885904b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:43.993377 12211 leader_election.cc:290] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:43.995637 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d356fcc48441ca80f2df29b533404f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:43.996331 10591 master_service.cc:960] ReplaceTablet: received request to replace tablet 50f45c02a6994c5aaaf9c0a499e15fed from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:43.996784 12291 raft_consensus.cc:2749] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:43.998337 10805 leader_election.cc:343] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.001582 11283 leader_election.cc:304] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
W20260809 01:45:44.003212 11283 leader_election.cc:343] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: b870347b07b04de5b833ab4fd3a2475f
I20260809 01:45:44.003806 11283 leader_election.cc:304] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:44.006850 11949 ts_tablet_manager.cc:1929] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:44.007189 11949 log.cc:1199] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/7501d985958746bea58e0ff752d7fed8
I20260809 01:45:44.007769 11949 ts_tablet_manager.cc:1950] T 7501d985958746bea58e0ff752d7fed8 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:44.007901 12276 raft_consensus.cc:2804] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:44.008209 12276 raft_consensus.cc:493] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:44.008560 12276 raft_consensus.cc:3060] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:44.009851 11553 tablet_bootstrap.cc:492] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:44.010353 11553 ts_tablet_manager.cc:1403] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.115s user 0.006s sys 0.009s
I20260809 01:45:44.012732 11553 raft_consensus.cc:359] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.013365 11553 raft_consensus.cc:385] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:44.018798 11553 raft_consensus.cc:740] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:44.025151 11553 consensus_queue.cc:260] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.027326 11553 ts_tablet_manager.cc:1434] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260809 01:45:44.032618 11152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "995a37a9e6b14889b876da5d76c89b84" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:44.033653 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 136) took 4752 ms. Trace:
I20260809 01:45:44.026736 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8baa4ae62edf46b5a386661a6826bbab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.033932 12266 raft_consensus.cc:2749] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:45:44.033993 11949 rpcz_store.cc:276] 0809 01:45:39.281156 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.281316 (+ 160us) service_pool.cc:224] Handling call
0809 01:45:43.918337 (+4637021us) ts_tablet_manager.cc:1151] Deleting tablet 7501d985958746bea58e0ff752d7fed8
0809 01:45:43.918376 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:43.961325 (+ 42949us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.006739 (+ 45414us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.033588 (+ 26849us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4636529}
I20260809 01:45:44.038226 11553 tablet_bootstrap.cc:492] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:44.043215 11553 tablet_bootstrap.cc:654] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:44.048902 11152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6159d660d544b3e907a08372027c0e2" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:43.984685 11342 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 179) took 1568 ms. Trace:
I20260809 01:45:43.962203 11337 tablet_service.cc:1511] Processing CreateTablet for tablet 31874b7e70a847a4885bd09da40a6902 (DEFAULT_TABLE table=default.Table_53624396796f4684acbc3787e6e05f6e [id=18e87d9e1fa3442aaa489e3abcd4640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.055111 11949 tablet_replica.cc:333] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:44.047966 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197b5043f6d348f89853c073b256daa4" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:44.065492 11152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7803a6aff02d42c1833f9b2cd7332b9f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:44.068478 11152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d356fcc48441ca80f2df29b533404f" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:44.069779 10804 leader_election.cc:343] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.070147 10804 leader_election.cc:304] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.070816 12211 raft_consensus.cc:2749] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:44.071491 10804 leader_election.cc:343] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.071905 10804 leader_election.cc:304] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.071724 11949 raft_consensus.cc:2243] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:44.074522 12211 raft_consensus.cc:2749] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:44.077032 11949 raft_consensus.cc:2272] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:44.080047 11361 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 169) took 2050 ms. Trace:
W20260809 01:45:44.080389 10581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111) [suppressed 4 similar messages]
I20260809 01:45:44.080468 11361 rpcz_store.cc:276] 0809 01:45:42.029967 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.384286 (+354319us) service_pool.cc:224] Handling call
0809 01:45:43.121671 (+737385us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.121750 (+ 79us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.249301 (+127551us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.776929 (+527628us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.079977 (+303048us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":133491}
I20260809 01:45:44.082027 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 058dfc9b37f04fd8a72981b4290390b4 (DEFAULT_TABLE table=default.Table_a97d9950868e4ee0b869484bdde6827f [id=0cff66be49af4f69b1ddb65d69af11c2]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:44.062152 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 123296us
I20260809 01:45:44.086973 11949 ts_tablet_manager.cc:1916] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:43.968873 11352 rpcz_store.cc:276] 0809 01:45:42.415418 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.520048 (+104630us) service_pool.cc:224] Handling call
0809 01:45:43.118082 (+598034us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.118178 (+ 96us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.224801 (+106623us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.854291 (+629490us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.967849 (+113558us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19c68 00007f0ee1f19b8b 00007f0eeae871d9 00007f0eea09f4e8 00007f0eea165783 00007f0eea16151f 00007f0eea161cdd 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82
0809 01:45:43.968258 (+ 409us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":137288,"spinlock_wait_cycles":13268864}
I20260809 01:45:44.097755 11352 tablet_service.cc:1558] Processing DeleteTablet for tablet c3bf43978dc94c2b9a7f3c18a4677c40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:43 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:44.097966 11553 tablet_bootstrap.cc:492] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:44.098388 11553 ts_tablet_manager.cc:1403] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.060s user 0.014s sys 0.009s
I20260809 01:45:44.098488 11352 tablet_service.cc:1558] Processing DeleteTablet for tablet 403cb09c1786491ab0d40502038bf9c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:43 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:44.102193 11553 raft_consensus.cc:359] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.055122 11342 rpcz_store.cc:276] 0809 01:45:42.416712 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.895556 (+478844us) service_pool.cc:224] Handling call
0809 01:45:43.130612 (+235056us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.130757 (+ 145us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.331255 (+200498us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.965976 (+634721us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.984244 (+ 18268us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f19b52 00007f0eeae871d9 00007f0eea09f4e8 00007f0eea165783 00007f0eea16151f 00007f0eea161cdd 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82
0809 01:45:43.984615 (+ 371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39704320}
I20260809 01:45:44.109915 11364 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 187) took 1106 ms. Trace:
W20260809 01:45:44.105382 10804 leader_election.cc:343] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.110560 10804 leader_election.cc:304] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.111119 12211 raft_consensus.cc:2749] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:44.102815 11553 raft_consensus.cc:385] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:44.112278 11553 raft_consensus.cc:740] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:44.110278 11364 rpcz_store.cc:276] 0809 01:45:43.003715 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.123118 (+119403us) service_pool.cc:224] Handling call
0809 01:45:43.124213 (+ 1095us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.124307 (+ 94us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.217233 (+ 92926us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.776162 (+558929us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.109860 (+333698us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":287554}
I20260809 01:45:44.115630 11553 consensus_queue.cc:260] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:44.124080 10804 leader_election.cc:343] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.124524 10804 leader_election.cc:304] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.125229 12211 raft_consensus.cc:2749] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:43.974579 11351 tablet_service.cc:1511] Processing CreateTablet for tablet 9465f8bb39d64e1cab8646282cb65c75 (DEFAULT_TABLE table=default.Table_72e975bfbe594052a24baee87785c71a [id=04d4b772004b44e781c255cf271fab34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.128476 11553 ts_tablet_manager.cc:1434] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.030s user 0.005s sys 0.000s
I20260809 01:45:43.993151 11353 tablet_service.cc:1511] Processing CreateTablet for tablet b3b02bd8c93b43bf90ff0192a9541aae (DEFAULT_TABLE table=default.Table_085fd4536b524eb9aacb353d95ffa166 [id=c0797cd687f2480087ce8379df004967]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.137622 11346 tablet_service.cc:1511] Processing CreateTablet for tablet c704d148da8d406aa824253a51b850c8 (DEFAULT_TABLE table=default.Table_29e8714d4cdd4d308c5856c9b672239d [id=c064685015ef4b88ac9132f920977a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.137478 11553 tablet_bootstrap.cc:492] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:44.152926 11949 ts_tablet_manager.cc:1929] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:44.157508 11949 log.cc:1199] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/a9d1ed215ba644abbae09e78016b2c21
I20260809 01:45:44.158874 11949 ts_tablet_manager.cc:1950] T a9d1ed215ba644abbae09e78016b2c21 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:44.160673 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 137) took 4842 ms. Trace:
I20260809 01:45:44.157215 11553 tablet_bootstrap.cc:654] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:44.161001 11949 rpcz_store.cc:276] 0809 01:45:39.317960 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.318230 (+ 270us) service_pool.cc:224] Handling call
0809 01:45:44.036553 (+4718323us) ts_tablet_manager.cc:1151] Deleting tablet a9d1ed215ba644abbae09e78016b2c21
0809 01:45:44.054994 (+ 18441us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.131441 (+ 76447us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.152820 (+ 21379us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.160620 (+ 7800us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4717664}
I20260809 01:45:44.165997 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet a9d1ed215ba644abbae09e78016b2c21 (table default.Table_e111eb8aa0754bdd9e6e571b06d79add [id=78fdf345e656424e9612369430887930]) successfully deleted
I20260809 01:45:44.166320 11949 tablet_replica.cc:333] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:44.171522 11949 raft_consensus.cc:2243] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:44.171978 11949 raft_consensus.cc:2272] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:44.174409 11949 ts_tablet_manager.cc:1916] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:44.194806 11949 ts_tablet_manager.cc:1929] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260809 01:45:44.195088 10589 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c87361b29664464f9fd73be974e2d3a0 consensus state has no leader
I20260809 01:45:44.195150 11949 log.cc:1199] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/c2f86bba4bd94fbcacdc9c139e9de2d7
I20260809 01:45:44.202193 11949 ts_tablet_manager.cc:1950] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:44.204388 11553 tablet_bootstrap.cc:492] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:44.204862 11553 ts_tablet_manager.cc:1403] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.068s user 0.003s sys 0.024s
I20260809 01:45:44.208371 11553 raft_consensus.cc:359] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.210152 11553 raft_consensus.cc:385] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:44.210458 11553 raft_consensus.cc:740] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:44.209784 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 138) took 4881 ms. Trace:
I20260809 01:45:44.211095 11949 rpcz_store.cc:276] 0809 01:45:39.328888 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.329033 (+ 145us) service_pool.cc:224] Handling call
0809 01:45:44.166157 (+4837124us) ts_tablet_manager.cc:1151] Deleting tablet c2f86bba4bd94fbcacdc9c139e9de2d7
0809 01:45:44.166200 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.181705 (+ 15505us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.194707 (+ 13002us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.209726 (+ 15019us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4836527}
I20260809 01:45:44.213600 11949 tablet_replica.cc:333] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:44.213649 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet c2f86bba4bd94fbcacdc9c139e9de2d7 (table default.Table_5f4d617a86c2487fa9f64cb58f14de3c [id=f64daa5a2706436081e9166a5f3a0cd7]) successfully deleted
I20260809 01:45:44.212118 11553 consensus_queue.cc:260] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:44.215004 10804 leader_election.cc:343] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.215407 10804 leader_election.cc:304] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.219942 12211 raft_consensus.cc:2749] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:44.222720 11553 ts_tablet_manager.cc:1434] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.018s user 0.001s sys 0.005s
I20260809 01:45:44.227741 11949 raft_consensus.cc:2243] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:44.228394 11949 raft_consensus.cc:2272] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:43.984746 11154 rpcz_store.cc:276] 0809 01:45:41.656022 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.656209 (+ 187us) service_pool.cc:224] Handling call
0809 01:45:41.656622 (+ 413us) raft_consensus.cc:1471] Updating replica for 1 ops
0809 01:45:41.660184 (+ 3562us) raft_consensus.cc:1534] Early marking committed up to index 0
0809 01:45:41.660223 (+ 39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0809 01:45:41.668591 (+ 8368us) alter_schema_op.cc:128] START. Timestamp: 1786239940534612
0809 01:45:42.160327 (+491736us) log.cc:844] Serialized 173 byte log entry
0809 01:45:42.607390 (+447063us) raft_consensus.cc:1645] Marking committed up to 4
0809 01:45:43.983532 (+1376142us) raft_consensus.cc:1697] Filling consensus response to leader.
0809 01:45:43.983665 (+ 133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0809 01:45:43.983736 (+ 71us) raft_consensus.cc:1685] finished
0809 01:45:43.983784 (+ 48us) raft_consensus.cc:1691] UpdateReplicas() finished
0809 01:45:43.984011 (+ 227us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:42.025646 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0809 01:45:42.026121 (+ 475us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 01:45:42.026152 (+ 31us) alter_schema_op.cc:66] Acquired schema lock
0809 01:45:42.026206 (+ 54us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0809 01:45:42.670327 (+644121us) op_driver.cc:464] REPLICATION: finished
0809 01:45:43.669940 (+999613us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0809 01:45:43.757922 (+ 87982us) log.cc:844] Serialized 5 byte log entry
Metrics: {"thread_start_us":138103,"threads_started":1,"wal-append.queue_time_us":141878,"wal-append.run_cpu_time_us":2348,"wal-append.run_wall_time_us":1293745,"child_traces":[["op",{"apply.queue_time_us":951317,"prepare.queue_time_us":356806,"prepare.run_cpu_time_us":620,"prepare.run_wall_time_us":620,"replication_time_us":957445,"thread_start_us":1477081,"threads_started":3}]]}
I20260809 01:45:44.235409 11949 ts_tablet_manager.cc:1916] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:44.234452 11553 tablet_bootstrap.cc:492] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:44.242208 11113 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 135) took 3738 ms. Trace:
I20260809 01:45:44.242776 11113 rpcz_store.cc:276] 0809 01:45:40.503551 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.504375 (+ 824us) service_pool.cc:224] Handling call
0809 01:45:41.233059 (+728684us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.233250 (+ 191us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.475612 (+2242362us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.151298 (+675686us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.241425 (+ 90127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":354299}
I20260809 01:45:44.246150 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 107) took 6266 ms. Trace:
I20260809 01:45:44.246472 11951 rpcz_store.cc:276] 0809 01:45:37.979345 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:37.980366 (+ 1021us) service_pool.cc:224] Handling call
0809 01:45:43.129776 (+5149410us) ts_tablet_manager.cc:1151] Deleting tablet 5f8824953c794450ba274b160d891fcb
0809 01:45:43.133970 (+ 4194us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:43.264919 (+130949us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.733044 (+468125us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.246083 (+513039us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":149658,"tablet-delete.queue_time_us":5148822}
I20260809 01:45:44.250059 11553 tablet_bootstrap.cc:654] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:44.252885 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 5f8824953c794450ba274b160d891fcb (table default.Table_636659c489a74b76ab839543e432ee5c [id=64ba63dd9d4c4c76bd811181b3ee062e]) successfully deleted
I20260809 01:45:44.262006 11116 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 132) took 3801 ms. Trace:
I20260809 01:45:44.262355 11116 rpcz_store.cc:276] 0809 01:45:40.460670 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.465763 (+ 5093us) service_pool.cc:224] Handling call
0809 01:45:41.176026 (+710263us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.217726 (+ 41700us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.623765 (+2406039us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.219525 (+595760us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.261861 (+ 42336us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103951}
I20260809 01:45:43.897190 11117 tablet_service.cc:1511] Processing CreateTablet for tablet 34b334f049e24768ad2291326ae70121 (DEFAULT_TABLE table=default.Table_824915c2101a47bd9948a0d422cc8d16 [id=d8c7495a8995426c847fb77372c5e95a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.263994 11116 tablet_service.cc:1511] Processing CreateTablet for tablet 7770aee1b7af415d995f63672fd05c01 (DEFAULT_TABLE table=default.Table_84167a58d3634ac7b44df384d0b7288f [id=d1ca00e3d6234799a396d40b3789e9f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.264333 11117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34b334f049e24768ad2291326ae70121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.264429 11949 ts_tablet_manager.cc:1929] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:44.264734 11949 log.cc:1199] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/dc230f25baf94cbf80aa8e2771ec23ba
I20260809 01:45:44.264976 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7770aee1b7af415d995f63672fd05c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.265312 11949 ts_tablet_manager.cc:1950] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:44.268021 12222 ts_tablet_manager.cc:1929] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:44.268428 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 139) took 4697 ms. Trace:
I20260809 01:45:44.268448 12222 log.cc:1199] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/737d7047a0c74a03b3203112987c0cb2
I20260809 01:45:44.268678 11949 rpcz_store.cc:276] 0809 01:45:39.571518 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.571648 (+ 130us) service_pool.cc:224] Handling call
0809 01:45:44.213437 (+4641789us) ts_tablet_manager.cc:1151] Deleting tablet dc230f25baf94cbf80aa8e2771ec23ba
0809 01:45:44.213479 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.253227 (+ 39748us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.264320 (+ 11093us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.268377 (+ 4057us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4641304}
I20260809 01:45:44.273681 11553 tablet_bootstrap.cc:492] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:44.274161 11553 ts_tablet_manager.cc:1403] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.040s user 0.014s sys 0.002s
W20260809 01:45:44.277310 10804 leader_election.cc:343] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.277693 10804 leader_election.cc:304] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:44.278194 10804 raft_consensus.cc:2686] T dc230f25baf94cbf80aa8e2771ec23ba P b5117038a52a45b0aea3d30ca9cfcbe8: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260809 01:45:44.279739 10804 leader_election.cc:343] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.280063 10804 leader_election.cc:304] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.280709 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet dc230f25baf94cbf80aa8e2771ec23ba (table default.Table_1998ec81597a4c4584dc4e4f90caa274 [id=cb347b72c1cf436095c6043c5b169641]) successfully deleted
I20260809 01:45:44.270838 11949 tablet_replica.cc:333] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:44.281860 11949 raft_consensus.cc:2243] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:44.282542 11949 raft_consensus.cc:2272] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:44.276955 11553 raft_consensus.cc:359] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.284732 11553 raft_consensus.cc:385] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:44.285037 11553 raft_consensus.cc:740] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:44.285809 11553 consensus_queue.cc:260] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.259815 11554 tablet_bootstrap.cc:492] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:44.286844 11949 ts_tablet_manager.cc:1916] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:44.291664 11554 ts_tablet_manager.cc:1403] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 1.095s user 0.007s sys 0.008s
I20260809 01:45:44.293586 11553 ts_tablet_manager.cc:1434] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.019s user 0.004s sys 0.002s
I20260809 01:45:44.296833 12211 raft_consensus.cc:493] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:44.297282 12211 raft_consensus.cc:515] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.299325 12211 leader_election.cc:290] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:44.303200 11553 tablet_bootstrap.cc:492] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:44.269346 12222 ts_tablet_manager.cc:1950] T 737d7047a0c74a03b3203112987c0cb2 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:44.270314 11111 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 137) took 3752 ms. Trace:
I20260809 01:45:44.274868 11115 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 133) took 3793 ms. Trace:
W20260809 01:45:44.306439 10804 raft_consensus.cc:2686] T c2f86bba4bd94fbcacdc9c139e9de2d7 P b5117038a52a45b0aea3d30ca9cfcbe8: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260809 01:45:44.310639 10804 leader_election.cc:343] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.311019 10804 leader_election.cc:304] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.311697 12211 raft_consensus.cc:2749] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:44.313076 10804 leader_election.cc:343] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.313479 10804 leader_election.cc:304] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.315503 12211 raft_consensus.cc:2749] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:44.316991 10804 leader_election.cc:343] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.318964 10804 leader_election.cc:304] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.319744 12211 raft_consensus.cc:2749] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:44.322108 10804 leader_election.cc:343] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.325001 10804 leader_election.cc:304] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.325807 12211 raft_consensus.cc:2749] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:44.328426 10804 leader_election.cc:343] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.329998 10804 leader_election.cc:304] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.334216 12211 raft_consensus.cc:2749] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:44.336397 10804 leader_election.cc:343] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.337007 10804 leader_election.cc:304] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.337728 12211 raft_consensus.cc:2749] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:44.338610 10804 leader_election.cc:343] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.338986 10804 leader_election.cc:304] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.344907 12211 raft_consensus.cc:2749] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:44.345973 10804 leader_election.cc:343] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.346357 10804 leader_election.cc:304] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.346975 12211 raft_consensus.cc:2749] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:44.347836 10804 leader_election.cc:343] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.348215 10804 leader_election.cc:304] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.349453 12211 raft_consensus.cc:2749] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:44.350805 10804 leader_election.cc:343] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.351320 10804 leader_election.cc:304] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.351990 12211 raft_consensus.cc:2749] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:44.352895 11553 tablet_bootstrap.cc:654] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:44.344403 11949 ts_tablet_manager.cc:1929] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:45:44.360630 11949 log.cc:1199] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/44ac6aecc93447f7935fbd656fd2a86b
I20260809 01:45:44.361498 11949 ts_tablet_manager.cc:1950] T 44ac6aecc93447f7935fbd656fd2a86b P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
W20260809 01:45:44.366240 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 363055us
I20260809 01:45:44.274974 11105 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 149) took 2625 ms. Trace:
I20260809 01:45:44.275559 11124 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 152) took 2583 ms. Trace:
I20260809 01:45:44.369871 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 140) took 4529 ms. Trace:
I20260809 01:45:44.370149 11949 rpcz_store.cc:276] 0809 01:45:39.840221 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.840343 (+ 122us) service_pool.cc:224] Handling call
0809 01:45:44.270708 (+4430365us) ts_tablet_manager.cc:1151] Deleting tablet 44ac6aecc93447f7935fbd656fd2a86b
0809 01:45:44.270742 (+ 34us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.331679 (+ 60937us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.344280 (+ 12601us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.369819 (+ 25539us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4429897}
I20260809 01:45:44.282972 12246 tablet.cc:1724] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:44.377429 11949 tablet_replica.cc:333] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:44.284313 11114 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 134) took 3786 ms. Trace:
I20260809 01:45:44.289052 11123 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 157) took 1903 ms. Trace:
I20260809 01:45:44.378243 11949 raft_consensus.cc:2243] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:44.299093 11120 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 155) took 2133 ms. Trace:
I20260809 01:45:44.378962 11949 raft_consensus.cc:2272] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:44.380749 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 44ac6aecc93447f7935fbd656fd2a86b (table default.Table_57234dd818414b7397d1ac621bdb06a7 [id=17cbc8bf77e142d08d03bd7a051e00ad]) successfully deleted
I20260809 01:45:44.381240 11949 ts_tablet_manager.cc:1916] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:44.377964 11114 rpcz_store.cc:276] 0809 01:45:40.498114 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.498299 (+ 185us) service_pool.cc:224] Handling call
0809 01:45:41.231210 (+732911us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.231359 (+ 149us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.631556 (+2400197us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.240828 (+609272us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.284149 (+ 43321us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":196256}
I20260809 01:45:44.387941 11114 tablet_service.cc:1511] Processing CreateTablet for tablet 14b1d03085eb4ab69789fed6fab882ae (DEFAULT_TABLE table=default.Table_25d50c77e3f5425888afe07872b75188 [id=f5a337b5983945e7bc23ddd3818dfff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.378079 11123 rpcz_store.cc:276] 0809 01:45:42.433223 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.606500 (+1173277us) service_pool.cc:224] Handling call
0809 01:45:43.612635 (+ 6135us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.612709 (+ 74us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.192089 (+579380us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.288615 (+ 96526us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.288986 (+ 371us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:44.378793 11120 rpcz_store.cc:276] 0809 01:45:42.227900 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.283557 (+ 55657us) service_pool.cc:224] Handling call
0809 01:45:42.287021 (+ 3464us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.287101 (+ 80us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.998473 (+1711372us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.298350 (+299877us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.298759 (+ 409us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:44.390661 11120 tablet_service.cc:1511] Processing CreateTablet for tablet 100ad529186e4821b85ba28996933a8a (DEFAULT_TABLE table=default.Table_3aab7a7876fe46caadaed8dde7387416 [id=67b5f34f642646319bbfa3de7ed8895d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.392678 11123 tablet_service.cc:1511] Processing CreateTablet for tablet 00c7ec3928224059ab7f58d203224536 (DEFAULT_TABLE table=default.Table_753471f1b2c1464c8ed9a6591a0ebd9e [id=374e6a771a36443e9ecf07563041b587]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.395219 11553 tablet_bootstrap.cc:492] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:44.395699 11553 ts_tablet_manager.cc:1403] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.093s user 0.006s sys 0.010s
I20260809 01:45:44.369211 11124 rpcz_store.cc:276] 0809 01:45:41.691828 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.065340 (+373512us) service_pool.cc:224] Handling call
0809 01:45:42.066390 (+ 1050us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.066462 (+ 72us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.650220 (+1583758us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.191780 (+541560us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.275389 (+ 83609us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82513}
I20260809 01:45:44.397230 11949 ts_tablet_manager.cc:1929] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:45:44.397655 11949 log.cc:1199] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/c9d839443474498386220c288ff1721b
I20260809 01:45:44.398397 11949 ts_tablet_manager.cc:1950] T c9d839443474498386220c288ff1721b P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:44.399191 11553 raft_consensus.cc:359] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.400027 11553 raft_consensus.cc:385] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:44.400329 11553 raft_consensus.cc:740] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:44.400411 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 160) took 3371 ms. Trace:
I20260809 01:45:44.400986 11110 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 138) took 3875 ms. Trace:
I20260809 01:45:44.401795 11553 consensus_queue.cc:260] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.401090 11949 rpcz_store.cc:276] 0809 01:45:41.029037 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.029169 (+ 132us) service_pool.cc:224] Handling call
0809 01:45:44.377286 (+3348117us) ts_tablet_manager.cc:1151] Deleting tablet c9d839443474498386220c288ff1721b
0809 01:45:44.377322 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.390369 (+ 13047us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.397127 (+ 6758us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.400345 (+ 3218us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3347391}
I20260809 01:45:44.404284 11949 tablet_replica.cc:333] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:44.405380 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet c9d839443474498386220c288ff1721b (table default.Table_0bcdcc37d5f04f40ab4dd02b66baf4a6 [id=97ccdb35be6a437288222feb89f9e751]) successfully deleted
W20260809 01:45:44.406292 10580 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet c87361b29664464f9fd73be974e2d3a0 (table default.Table_0708cade7ac74a54aedeb8fe16b3b8df [id=74e764815e2242a9be1ca850a0b6ff30], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet c87361b29664464f9fd73be974e2d3a0 consensus state has no leader
I20260809 01:45:44.407141 11949 raft_consensus.cc:2243] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:44.407538 11949 raft_consensus.cc:2272] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:44.409219 11949 ts_tablet_manager.cc:1916] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:44.406410 11553 ts_tablet_manager.cc:1434] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.010s user 0.001s sys 0.005s
I20260809 01:45:44.413262 11553 tablet_bootstrap.cc:492] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:44.413230 10589 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_e53beaa5c2fe49e3ba6eaa8181b5106a" } new_table_name: "default.Table_c54e4ecef7c145cc970af6169c5a9326" modify_external_catalogs: true
I20260809 01:45:44.429893 11553 tablet_bootstrap.cc:654] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:44.437966 11949 ts_tablet_manager.cc:1929] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:44.438319 11949 log.cc:1199] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/60d356fcc48441ca80f2df29b533404f
I20260809 01:45:44.372437 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9493892c1ff48e7a0558ff0fcb7a0f3" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:44.439127 11949 ts_tablet_manager.cc:1950] T 60d356fcc48441ca80f2df29b533404f P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
W20260809 01:45:44.440475 10805 leader_election.cc:343] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.441090 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 166) took 2108 ms. Trace:
I20260809 01:45:44.382696 11336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 189) took 1378 ms. Trace:
I20260809 01:45:44.426905 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 058dfc9b37f04fd8a72981b4290390b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.442989 11949 rpcz_store.cc:276] 0809 01:45:42.332904 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.333451 (+ 547us) service_pool.cc:224] Handling call
0809 01:45:44.404138 (+2070687us) ts_tablet_manager.cc:1151] Deleting tablet 60d356fcc48441ca80f2df29b533404f
0809 01:45:44.404179 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.422634 (+ 18455us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.437860 (+ 15226us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.441015 (+ 3155us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2069818}
I20260809 01:45:44.443771 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e360e79820d148f4a6e9fdeb2c19da54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.429364 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 108) took 6443 ms. Trace:
I20260809 01:45:44.445762 11553 tablet_bootstrap.cc:492] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:44.446879 11553 ts_tablet_manager.cc:1403] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.034s user 0.010s sys 0.006s
I20260809 01:45:44.445039 11951 rpcz_store.cc:276] 0809 01:45:37.985764 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:37.985885 (+ 121us) service_pool.cc:224] Handling call
0809 01:45:44.247916 (+6262031us) ts_tablet_manager.cc:1151] Deleting tablet 48b7459ad0974cb9948d647459b22260
0809 01:45:44.378167 (+130251us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f19b52 00007f0eeae871d9 00007f0eea09f4e8 00007f0eea165aa9 00007f0eea1660a3 00007f0eea1970e0 00007f0eea17d64d 00007f0eea17d5f9 00007f0eea17d589
0809 01:45:44.428453 (+ 50286us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.429077 (+ 624us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296095104,"tablet-delete.queue_time_us":6261447}
I20260809 01:45:44.447561 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 60d356fcc48441ca80f2df29b533404f (table default.Table_29a4bd6bd10a473dbd9dbda008b9eb16 [id=67ee87fabc7749279341528fbcce16d4]) successfully deleted
W20260809 01:45:44.448491 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 48b7459ad0974cb9948d647459b22260 because the tablet was not found. No further retry: Not found: Tablet not found: 48b7459ad0974cb9948d647459b22260
I20260809 01:45:44.450035 11337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31874b7e70a847a4885bd09da40a6902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.450776 11553 raft_consensus.cc:359] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.451665 11553 raft_consensus.cc:385] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:44.451906 11553 raft_consensus.cc:740] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:44.452610 11553 consensus_queue.cc:260] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.453653 11949 tablet_replica.cc:333] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:44.454371 11949 raft_consensus.cc:2243] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:44.455216 11949 raft_consensus.cc:2272] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:44.456802 11949 ts_tablet_manager.cc:1916] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:44.462458 11553 ts_tablet_manager.cc:1434] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
I20260809 01:45:44.463332 11553 tablet_bootstrap.cc:492] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:44.472066 11553 tablet_bootstrap.cc:654] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:44.403373 11551 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.527s user 0.032s sys 0.040s
I20260809 01:45:44.486397 11949 ts_tablet_manager.cc:1929] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:44.486819 11949 log.cc:1199] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/995a37a9e6b14889b876da5d76c89b84
I20260809 01:45:44.487491 11949 ts_tablet_manager.cc:1950] T 995a37a9e6b14889b876da5d76c89b84 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:44.489300 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 167) took 2154 ms. Trace:
I20260809 01:45:44.304572 11111 rpcz_store.cc:276] 0809 01:45:40.517805 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.518298 (+ 493us) service_pool.cc:224] Handling call
0809 01:45:41.236042 (+717744us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.236169 (+ 127us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.658172 (+2422003us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.216919 (+558747us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.270113 (+ 53194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":170176}
I20260809 01:45:44.489653 11949 rpcz_store.cc:276] 0809 01:45:42.334662 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.335098 (+ 436us) service_pool.cc:224] Handling call
0809 01:45:44.453497 (+2118399us) ts_tablet_manager.cc:1151] Deleting tablet 995a37a9e6b14889b876da5d76c89b84
0809 01:45:44.453553 (+ 56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.475948 (+ 22395us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.486290 (+ 10342us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.489247 (+ 2957us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":418,"tablet-delete.queue_time_us":2117825}
I20260809 01:45:44.369076 11105 rpcz_store.cc:276] 0809 01:45:41.649538 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.649644 (+ 106us) service_pool.cc:224] Handling call
0809 01:45:41.742712 (+ 93068us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.752253 (+ 9541us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.632557 (+1880304us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.212193 (+579636us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.274807 (+ 62614us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":238253}
I20260809 01:45:44.368209 11115 rpcz_store.cc:276] 0809 01:45:40.481340 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.481496 (+ 156us) service_pool.cc:224] Handling call
0809 01:45:41.226977 (+745481us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.230936 (+ 3959us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.612280 (+2381344us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.190063 (+577783us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.274694 (+ 84631us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":339105}
I20260809 01:45:44.490934 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 995a37a9e6b14889b876da5d76c89b84 (table default.Table_50f4db524f554499a2dab6d3bc48f053 [id=85ee30d1d77d47f691e6a0dd498d9bca]) successfully deleted
I20260809 01:45:44.491160 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 169) took 2146 ms. Trace:
I20260809 01:45:44.454931 11124 tablet_service.cc:1511] Processing CreateTablet for tablet e360e79820d148f4a6e9fdeb2c19da54 (DEFAULT_TABLE table=default.Table_7f59bdaa42d240d681f6b296de78b329 [id=e800768eb8a2407c93290b889cc5f1a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.491487 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 109) took 6487 ms. Trace:
I20260809 01:45:44.396873 11113 tablet_service.cc:1511] Processing CreateTablet for tablet 3938bfd21b294fa6a25f0d8efc1ca29a (DEFAULT_TABLE table=default.Table_86ddcc80418e437f8fb35426024f6d19 [id=01721069a381465397d92855a7accc23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.491460 11949 rpcz_store.cc:276] 0809 01:45:42.344816 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.345242 (+ 426us) service_pool.cc:224] Handling call
0809 01:45:44.490938 (+2145696us) ts_tablet_manager.cc:1151] Deleting tablet 12fb852b8ea8478cb160a265459865c4
0809 01:45:44.490976 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.491114 (+ 138us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2145206}
I20260809 01:45:44.491824 11951 rpcz_store.cc:276] 0809 01:45:38.003587 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.003713 (+ 126us) service_pool.cc:224] Handling call
0809 01:45:44.452957 (+6449244us) ts_tablet_manager.cc:1151] Deleting tablet 61c83b8bcf54461bb24565e3a989100a
0809 01:45:44.491177 (+ 38220us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.491422 (+ 245us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6448610}
I20260809 01:45:44.492803 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 170) took 2146 ms. Trace:
I20260809 01:45:44.493081 11949 rpcz_store.cc:276] 0809 01:45:42.346150 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.372397 (+ 26247us) service_pool.cc:224] Handling call
0809 01:45:44.492603 (+2120206us) ts_tablet_manager.cc:1151] Deleting tablet c795f81d2bc248eeb39b64d6aaa0370c
0809 01:45:44.492638 (+ 35us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.492761 (+ 123us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2117120}
W20260809 01:45:44.493613 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 61c83b8bcf54461bb24565e3a989100a because the tablet was not found. No further retry: Not found: Tablet not found: 61c83b8bcf54461bb24565e3a989100a
I20260809 01:45:44.494946 11365 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 170) took 2438 ms. Trace:
W20260809 01:45:44.495774 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 12fb852b8ea8478cb160a265459865c4 because the tablet was not found. No further retry: Not found: Tablet not found: 12fb852b8ea8478cb160a265459865c4
I20260809 01:45:44.495990 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 171) took 2120 ms. Trace:
W20260809 01:45:44.496429 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet c795f81d2bc248eeb39b64d6aaa0370c because the tablet was not found. No further retry: Not found: Tablet not found: c795f81d2bc248eeb39b64d6aaa0370c
I20260809 01:45:44.496307 11949 rpcz_store.cc:276] 0809 01:45:42.376013 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.377846 (+ 1833us) service_pool.cc:224] Handling call
0809 01:45:44.495759 (+2117913us) ts_tablet_manager.cc:1151] Deleting tablet 4754f2e2a35f4a6a91fa333e2cf564e4
0809 01:45:44.495798 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.495941 (+ 143us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2116670}
W20260809 01:45:44.497386 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 4754f2e2a35f4a6a91fa333e2cf564e4 because the tablet was not found. No further retry: Not found: Tablet not found: 4754f2e2a35f4a6a91fa333e2cf564e4
I20260809 01:45:44.465098 11114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14b1d03085eb4ab69789fed6fab882ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.498689 11553 tablet_bootstrap.cc:492] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:44.499140 11553 ts_tablet_manager.cc:1403] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.036s user 0.010s sys 0.004s
I20260809 01:45:44.502353 11553 raft_consensus.cc:359] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.503769 11553 raft_consensus.cc:385] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:44.504076 11553 raft_consensus.cc:740] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:44.503438 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 172) took 2123 ms. Trace:
I20260809 01:45:44.504772 11949 rpcz_store.cc:276] 0809 01:45:42.386881 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.386981 (+ 100us) service_pool.cc:224] Handling call
0809 01:45:44.503226 (+2116245us) ts_tablet_manager.cc:1151] Deleting tablet 737d7047a0c74a03b3203112987c0cb2
0809 01:45:44.503263 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.503393 (+ 130us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2115806}
W20260809 01:45:44.505990 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 737d7047a0c74a03b3203112987c0cb2 because the tablet was not found. No further retry: Not found: Tablet not found: 737d7047a0c74a03b3203112987c0cb2
I20260809 01:45:44.506316 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 173) took 2118 ms. Trace:
I20260809 01:45:44.507550 11949 rpcz_store.cc:276] 0809 01:45:42.387809 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.387940 (+ 131us) service_pool.cc:224] Handling call
0809 01:45:44.506037 (+2118097us) ts_tablet_manager.cc:1151] Deleting tablet ac6eded297214b58a5d8129a60ca1120
0809 01:45:44.506077 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.506265 (+ 188us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2117545}
I20260809 01:45:44.506870 11553 consensus_queue.cc:260] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:44.508720 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet ac6eded297214b58a5d8129a60ca1120 because the tablet was not found. No further retry: Not found: Tablet not found: ac6eded297214b58a5d8129a60ca1120
I20260809 01:45:44.510797 11553 ts_tablet_manager.cc:1434] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20260809 01:45:44.518389 11553 tablet_bootstrap.cc:492] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:44.525870 11949 tablet_replica.cc:333] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:44.526636 11949 raft_consensus.cc:2243] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:44.527065 11949 raft_consensus.cc:2272] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:44.528690 11949 ts_tablet_manager.cc:1916] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:44.536371 11553 tablet_bootstrap.cc:654] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:44.556686 11553 tablet_bootstrap.cc:492] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:44.557143 11553 ts_tablet_manager.cc:1403] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.039s user 0.010s sys 0.009s
I20260809 01:45:44.559725 11553 raft_consensus.cc:359] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.560520 11553 raft_consensus.cc:385] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:44.561084 11553 raft_consensus.cc:740] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:44.561939 11553 consensus_queue.cc:260] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.576196 11553 ts_tablet_manager.cc:1434] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20260809 01:45:44.575707 11365 rpcz_store.cc:276] 0809 01:45:42.056504 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.468958 (+412454us) service_pool.cc:224] Handling call
0809 01:45:42.846694 (+377736us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.868558 (+ 21864us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.794341 (+925783us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.109191 (+314850us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.397153 (+287962us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f19b52 00007f0eeae871d9 00007f0eea09f4e8 00007f0eea165783 00007f0eea16151f 00007f0eea161cdd 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82
0809 01:45:44.494841 (+ 97688us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67971840}
W20260809 01:45:44.434971 11289 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.10.8.124:33929, user_credentials={real_user=slave}} blocked reactor thread for 68620.8us
I20260809 01:45:44.578270 11949 ts_tablet_manager.cc:1929] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:44.578606 11949 log.cc:1199] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/c528b45de0d14c19af5d6836b7a14f1c
I20260809 01:45:44.579106 12211 raft_consensus.cc:493] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:44.579352 11949 ts_tablet_manager.cc:1950] T c528b45de0d14c19af5d6836b7a14f1c P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:44.579684 12211 raft_consensus.cc:515] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.581131 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 174) took 2192 ms. Trace:
I20260809 01:45:44.581432 11949 rpcz_store.cc:276] 0809 01:45:42.388853 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.388954 (+ 101us) service_pool.cc:224] Handling call
0809 01:45:44.511182 (+2122228us) ts_tablet_manager.cc:1151] Deleting tablet c528b45de0d14c19af5d6836b7a14f1c
0809 01:45:44.513357 (+ 2175us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.557927 (+ 44570us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.578156 (+ 20229us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.581077 (+ 2921us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2121711}
I20260809 01:45:44.582566 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet c528b45de0d14c19af5d6836b7a14f1c (table default.Table_53624396796f4684acbc3787e6e05f6e [id=18e87d9e1fa3442aaa489e3abcd4640e]) successfully deleted
I20260809 01:45:44.582770 11949 tablet_replica.cc:333] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:44.583480 11949 raft_consensus.cc:2243] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:44.584076 11949 raft_consensus.cc:2272] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:44.584774 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e03033672e947eab8cc143263281133" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:44.585867 11949 ts_tablet_manager.cc:1916] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:44.586295 12211 leader_election.cc:290] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:44.587260 11553 tablet_bootstrap.cc:492] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
W20260809 01:45:44.587977 10805 leader_election.cc:343] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.592455 11553 tablet_bootstrap.cc:654] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:44.430964 11333 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 172) took 2295 ms. Trace:
I20260809 01:45:44.604821 11553 tablet_bootstrap.cc:492] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:44.605243 11553 ts_tablet_manager.cc:1403] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.003s
I20260809 01:45:44.536836 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9465f8bb39d64e1cab8646282cb65c75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.607503 11553 raft_consensus.cc:359] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.608276 11553 raft_consensus.cc:385] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:44.608577 11553 raft_consensus.cc:740] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:44.609242 11553 consensus_queue.cc:260] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.611786 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c704d148da8d406aa824253a51b850c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.615931 11553 ts_tablet_manager.cc:1434] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260809 01:45:44.616516 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3b02bd8c93b43bf90ff0192a9541aae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.616772 11553 tablet_bootstrap.cc:492] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:44.623286 11553 tablet_bootstrap.cc:654] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:44.628599 11347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd3662c307ba4ff78b8f3165b23acb1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.635087 11949 ts_tablet_manager.cc:1929] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20260809 01:45:44.635628 11949 log.cc:1199] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/2aac67e7819f4a70b1050b558868f27f
I20260809 01:45:44.636627 11949 ts_tablet_manager.cc:1950] T 2aac67e7819f4a70b1050b558868f27f P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:44.598263 11333 rpcz_store.cc:276] 0809 01:45:42.136860 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.477933 (+341073us) service_pool.cc:224] Handling call
0809 01:45:42.880641 (+402708us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.889264 (+ 8623us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.117081 (+227817us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.918321 (+801240us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.379113 (+460792us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f19b52 00007f0eeae871d9 00007f0eea09f4e8 00007f0eea165783 00007f0eea16151f 00007f0eea161cdd 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82
0809 01:45:44.430809 (+ 51696us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117816,"spinlock_wait_cycles":607858048}
W20260809 01:45:44.647853 11360 consensus_meta.cc:287] T 60176118d61548ff96b001ab0dae7d21 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.579s user 0.006s sys 0.000s
I20260809 01:45:44.649092 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 175) took 2259 ms. Trace:
I20260809 01:45:44.649226 11360 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 191) took 1644 ms. Trace:
I20260809 01:45:44.649363 11949 rpcz_store.cc:276] 0809 01:45:42.389938 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.390070 (+ 132us) service_pool.cc:224] Handling call
0809 01:45:44.582634 (+2192564us) ts_tablet_manager.cc:1151] Deleting tablet 2aac67e7819f4a70b1050b558868f27f
0809 01:45:44.582671 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.592714 (+ 10043us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.634940 (+ 42226us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.649040 (+ 14100us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2192088}
I20260809 01:45:44.649492 11360 rpcz_store.cc:276] 0809 01:45:43.005022 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.633976 (+628954us) service_pool.cc:224] Handling call
0809 01:45:43.635352 (+ 1376us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.700745 (+ 65393us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.017257 (+316512us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.648814 (+631557us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.649176 (+ 362us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:44.650403 11949 tablet_replica.cc:333] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:44.546586 11349 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 186) took 1547 ms. Trace:
I20260809 01:45:44.651078 11949 raft_consensus.cc:2243] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:44.651098 10599 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 677cc81f739d44069b44b3921602e725
I20260809 01:45:44.651445 11949 raft_consensus.cc:2272] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:44.650966 11349 rpcz_store.cc:276] 0809 01:45:42.999565 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.075954 (+ 76389us) service_pool.cc:224] Handling call
0809 01:45:43.099894 (+ 23940us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f193d2 00007f0eeae5dfb9 00007f0eea09f640 00007f0eea184641 00007f0eea1618d3 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82 00007f0ee869cab1
0809 01:45:43.133547 (+ 33653us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.133646 (+ 99us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.031600 (+897954us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.117937 (+ 86337us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.545717 (+427780us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f19b52 00007f0eeae871d9 00007f0eea09f4e8 00007f0eea165783 00007f0eea16151f 00007f0eea161cdd 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82
0809 01:45:44.546495 (+ 778us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741152000}
I20260809 01:45:44.552978 11554 raft_consensus.cc:359] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:44.652215 11951 tablet_replica.cc:333] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:44.652146 11554 raft_consensus.cc:385] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:44.652510 11554 raft_consensus.cc:740] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:44.652992 11949 ts_tablet_manager.cc:1916] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:44.652972 11951 raft_consensus.cc:2243] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:44.653470 11554 consensus_queue.cc:260] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:44.496186 11336 rpcz_store.cc:276] 0809 01:45:43.004450 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.253824 (+249374us) service_pool.cc:224] Handling call
0809 01:45:43.255380 (+ 1556us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.255463 (+ 83us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.696131 (+440668us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.030434 (+334303us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.382608 (+352174us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260809 01:45:44.562923 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 81530.3us
W20260809 01:45:44.565482 12276 consensus_meta.cc:287] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.556s user 0.004s sys 0.006s
I20260809 01:45:44.566756 11345 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 183) took 1677 ms. Trace:
I20260809 01:45:44.657104 11345 rpcz_store.cc:276] 0809 01:45:42.888879 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.929286 (+ 40407us) service_pool.cc:224] Handling call
0809 01:45:43.128691 (+199405us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.132543 (+ 3852us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.999566 (+867023us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.566159 (+566593us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.566684 (+ 525us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260809 01:45:44.569741 11341 consensus_meta.cc:287] T 3938bfd21b294fa6a25f0d8efc1ca29a P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.687s user 0.005s sys 0.006s
I20260809 01:45:44.659039 11341 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 185) took 1663 ms. Trace:
I20260809 01:45:44.659483 11341 rpcz_store.cc:276] 0809 01:45:42.995773 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.996761 (+ 988us) service_pool.cc:224] Handling call
0809 01:45:43.133870 (+137109us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.137921 (+ 4051us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.882685 (+744764us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.658506 (+775821us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.658973 (+ 467us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:44.656311 12276 raft_consensus.cc:515] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:44.666749 11553 tablet_bootstrap.cc:492] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:44.667266 11553 ts_tablet_manager.cc:1403] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.051s user 0.011s sys 0.005s
I20260809 01:45:44.668290 12276 leader_election.cc:290] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:44.673625 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_8edd49df5953499184f2a39b02c69e91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:44.675297 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8edd49df5953499184f2a39b02c69e91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:44.442653 11125 tablet_service.cc:1511] Processing CreateTablet for tablet 60176118d61548ff96b001ab0dae7d21 (DEFAULT_TABLE table=default.Table_85b1c71bf0984d6bb91bbc04608af545 [id=b90c58e7b4a34ea2a22ec6e5aa4be490]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:44.695429 11127 consensus_meta.cc:287] T 19fe81e277cd4c798a07ceafa0b60566 P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 0.602s user 0.010s sys 0.000s
W20260809 01:45:44.490159 11551 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.885s user 0.033s sys 0.044s
I20260809 01:45:44.503767 11115 tablet_service.cc:1511] Processing CreateTablet for tablet f9a0fc14dd684ccaa19d51261b9c610e (DEFAULT_TABLE table=default.Table_33b1a3980a3f46b6bd99b4935671527c [id=2993a04ef2744f019c069da7ccc24532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.506105 11105 tablet_service.cc:1511] Processing CreateTablet for tablet bd3662c307ba4ff78b8f3165b23acb1a (DEFAULT_TABLE table=default.Table_f97a006f17844462abd4433315667e68 [id=0a517ec159a64c28aee43c00ca3f5359]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.508061 11111 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b (DEFAULT_TABLE table=default.Table_0bcdcc37d5f04f40ab4dd02b66baf4a6 [id=97ccdb35be6a437288222feb89f9e751]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:44.502882 11156 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.615s user 0.004s sys 0.055s
W20260809 01:45:44.696839 11156 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.918s user 0.055s sys 0.105s
I20260809 01:45:44.702955 11554 ts_tablet_manager.cc:1434] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.411s user 0.006s sys 0.002s
W20260809 01:45:44.532233 12279 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.774s user 0.004s sys 0.007s
I20260809 01:45:44.436040 11110 rpcz_store.cc:276] 0809 01:45:40.525461 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:40.525840 (+ 379us) service_pool.cc:224] Handling call
0809 01:45:41.237555 (+711715us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:41.238758 (+ 1203us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.004399 (+2765641us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.399284 (+394885us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.400264 (+ 980us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:44.711900 11554 tablet_bootstrap.cc:492] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:44.653403 11951 raft_consensus.cc:2272] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:44.712779 11110 tablet_service.cc:1558] Processing DeleteTablet for tablet b3b02bd8c93b43bf90ff0192a9541aae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:44.716387 11551 tablet_bootstrap.cc:492] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:44.716980 11551 ts_tablet_manager.cc:1403] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 2.840s user 0.046s sys 0.075s
I20260809 01:45:44.724972 11110 tablet_service.cc:1558] Processing DeleteTablet for tablet 761370ec8b164fd184a2e091030f610b with delete_type TABLET_DATA_DELETED (replaced by tablet 21e28316ad9442e2bcffba726d13cfe9) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:44.726225 11110 tablet_service.cc:1558] Processing DeleteTablet for tablet bd3662c307ba4ff78b8f3165b23acb1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:44.726289 11553 raft_consensus.cc:359] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:44.726992 11553 raft_consensus.cc:385] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:44.727255 11553 raft_consensus.cc:740] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:44.728614 11951 ts_tablet_manager.cc:1916] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:44.725884 11289 leader_election.cc:343] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: da86b5e4a27544c59ae6bb387440d246
W20260809 01:45:44.708483 12279 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.950s user 0.004s sys 0.008s
W20260809 01:45:44.735229 11289 consensus_peers.cc:597] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:44.738608 11369 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 198) took 1487 ms. Trace:
I20260809 01:45:44.738974 11369 rpcz_store.cc:276] 0809 01:45:43.251157 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.956151 (+704994us) service_pool.cc:224] Handling call
0809 01:45:43.982438 (+ 26287us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.982547 (+ 109us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.286611 (+304064us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.734805 (+448194us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.738559 (+ 3754us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:44.740473 11123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00c7ec3928224059ab7f58d203224536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.741676 11553 consensus_queue.cc:260] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:44.744004 11289 leader_election.cc:343] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: da86b5e4a27544c59ae6bb387440d246
I20260809 01:45:44.748634 11120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 100ad529186e4821b85ba28996933a8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.759627 11553 ts_tablet_manager.cc:1434] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.092s user 0.001s sys 0.005s
I20260809 01:45:44.760511 11553 tablet_bootstrap.cc:492] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:44.766227 11354 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 192) took 1760 ms. Trace:
I20260809 01:45:44.767314 12211 raft_consensus.cc:493] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:44.767712 12211 raft_consensus.cc:515] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:44.761662 12279 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 1.004s user 0.004s sys 0.008s
I20260809 01:45:44.769359 12211 leader_election.cc:290] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:44.769814 11354 rpcz_store.cc:276] 0809 01:45:43.005374 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.935108 (+929734us) service_pool.cc:224] Handling call
0809 01:45:43.936455 (+ 1347us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.936553 (+ 98us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.490622 (+554069us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.740347 (+249725us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.766171 (+ 25824us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:44.776495 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91941616dfdd4df3aaa7c8460876d273" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:44.779829 10804 leader_election.cc:343] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.790844 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 2aac67e7819f4a70b1050b558868f27f (table default.Table_72e975bfbe594052a24baee87785c71a [id=04d4b772004b44e781c255cf271fab34]) successfully deleted
I20260809 01:45:44.792701 12211 raft_consensus.cc:493] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:44.793130 12211 raft_consensus.cc:515] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.794615 11154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e03033672e947eab8cc143263281133" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:44.795197 12211 leader_election.cc:290] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:44.797021 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9493892c1ff48e7a0558ff0fcb7a0f3" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:44.797420 12211 raft_consensus.cc:493] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:44.797968 12211 raft_consensus.cc:515] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:44.803251 12211 leader_election.cc:290] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:44.804651 11949 ts_tablet_manager.cc:1929] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:44.805124 11949 log.cc:1199] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/5050a590fe0240ac8c9f9191ec2475a1
I20260809 01:45:44.810999 11949 ts_tablet_manager.cc:1950] T 5050a590fe0240ac8c9f9191ec2475a1 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:44.813169 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 176) took 2421 ms. Trace:
I20260809 01:45:44.816998 11554 tablet_bootstrap.cc:654] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:44.816793 11949 rpcz_store.cc:276] 0809 01:45:42.391816 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.391940 (+ 124us) service_pool.cc:224] Handling call
0809 01:45:44.650275 (+2258335us) ts_tablet_manager.cc:1151] Deleting tablet 5050a590fe0240ac8c9f9191ec2475a1
0809 01:45:44.650310 (+ 35us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.700231 (+ 49921us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.804486 (+104255us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.813090 (+ 8604us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2257726}
I20260809 01:45:44.816260 11553 tablet_bootstrap.cc:654] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:44.819034 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91941616dfdd4df3aaa7c8460876d273" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:44.820060 11949 tablet_replica.cc:333] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:44.820235 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f022e6a57a4ec0a5a8c33159e97a8e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:44.820755 11949 raft_consensus.cc:2243] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:44.821182 11949 raft_consensus.cc:2272] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:44.821324 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38bd344108649ed8e1687f82154000b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:44.822893 11949 ts_tablet_manager.cc:1916] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:44.823696 10805 leader_election.cc:343] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.824051 10805 leader_election.cc:304] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:44.824945 10805 leader_election.cc:343] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:44.825460 10805 leader_election.cc:343] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.831347 11553 tablet_bootstrap.cc:492] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:44.831784 11553 ts_tablet_manager.cc:1403] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.071s user 0.009s sys 0.001s
I20260809 01:45:44.837898 12211 raft_consensus.cc:2749] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:44.841142 11553 raft_consensus.cc:359] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.841848 11553 raft_consensus.cc:385] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:44.842162 11553 raft_consensus.cc:740] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:44.842870 11553 consensus_queue.cc:260] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:44.777431 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 76398.8us
I20260809 01:45:44.863294 11154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f022e6a57a4ec0a5a8c33159e97a8e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:44.865733 11551 raft_consensus.cc:359] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:44.866382 11551 raft_consensus.cc:385] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:44.866647 11551 raft_consensus.cc:740] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:44.867221 11551 consensus_queue.cc:260] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:44.868278 12211 raft_consensus.cc:493] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:44.868762 12211 raft_consensus.cc:515] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.870810 12211 leader_election.cc:290] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:44.853874 11110 tablet_service.cc:1511] Processing CreateTablet for tablet 02e2f77a70794ff7811b5f09d885904b (DEFAULT_TABLE table=default.Table_749b9564932646a498e0622597593241 [id=61aaf5db03ef43b0b02226dea5f1879c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.872582 11553 ts_tablet_manager.cc:1434] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.041s user 0.007s sys 0.000s
I20260809 01:45:44.873517 11553 tablet_bootstrap.cc:492] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:44.889804 11354 tablet_service.cc:1558] Processing DeleteTablet for tablet da86b5e4a27544c59ae6bb387440d246 with delete_type TABLET_DATA_DELETED (replaced by tablet 60d356fcc48441ca80f2df29b533404f) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:44.896533 11551 ts_tablet_manager.cc:1434] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.179s user 0.006s sys 0.002s
I20260809 01:45:44.897213 11553 tablet_bootstrap.cc:654] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:44.898066 11125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60176118d61548ff96b001ab0dae7d21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.899068 11127 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 161) took 1426 ms. Trace:
I20260809 01:45:44.899317 11127 rpcz_store.cc:276] 0809 01:45:43.472719 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.719062 (+246343us) service_pool.cc:224] Handling call
0809 01:45:43.744599 (+ 25537us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.762860 (+ 18261us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.092694 (+329834us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.731769 (+639075us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.899023 (+167254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:44.900677 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 31874b7e70a847a4885bd09da40a6902 (DEFAULT_TABLE table=default.Table_53624396796f4684acbc3787e6e05f6e [id=18e87d9e1fa3442aaa489e3abcd4640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.898572 11107 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 171) took 1423 ms. Trace:
I20260809 01:45:44.901875 11118 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 163) took 1428 ms. Trace:
I20260809 01:45:44.902215 11118 rpcz_store.cc:276] 0809 01:45:43.473354 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.728845 (+255491us) service_pool.cc:224] Handling call
0809 01:45:43.995428 (+266583us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.999704 (+ 4276us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.295687 (+295983us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.733709 (+438022us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.901824 (+168115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34948}
I20260809 01:45:44.904055 11949 ts_tablet_manager.cc:1929] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:44.903888 11118 tablet_service.cc:1511] Processing CreateTablet for tablet 9465f8bb39d64e1cab8646282cb65c75 (DEFAULT_TABLE table=default.Table_68ce5e1e513c4cbda72e059398b158a9 [id=04d4b772004b44e781c255cf271fab34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.904443 11949 log.cc:1199] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/b2726cb09f734ed18bb9379f94f3f22a
I20260809 01:45:44.904906 11551 tablet_bootstrap.cc:492] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:44.905076 11949 ts_tablet_manager.cc:1950] T b2726cb09f734ed18bb9379f94f3f22a P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:44.905154 11106 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 144) took 3282 ms. Trace:
I20260809 01:45:44.905471 11106 rpcz_store.cc:276] 0809 01:45:41.622490 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.632827 (+ 10337us) service_pool.cc:224] Handling call
0809 01:45:42.223395 (+590568us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.223563 (+ 168us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:43.845492 (+1621929us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.735457 (+889965us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.904987 (+169530us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23082}
I20260809 01:45:44.906471 11106 tablet_service.cc:1558] Processing DeleteTablet for tablet ef8a334741d3419eaa377338e0eaa889 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f771c8051784da18a6de9fcf45e0125) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:44.907464 11106 tablet_service.cc:1511] Processing CreateTablet for tablet c704d148da8d406aa824253a51b850c8 (DEFAULT_TABLE table=default.Table_29e8714d4cdd4d308c5856c9b672239d [id=c064685015ef4b88ac9132f920977a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.908493 11112 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 164) took 1434 ms. Trace:
I20260809 01:45:44.908793 11112 rpcz_store.cc:276] 0809 01:45:43.473671 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.818067 (+344396us) service_pool.cc:224] Handling call
0809 01:45:43.960798 (+142731us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.961038 (+ 240us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.484495 (+523457us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.751033 (+266538us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.908441 (+157408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:44.910228 11112 tablet_service.cc:1511] Processing CreateTablet for tablet b3b02bd8c93b43bf90ff0192a9541aae (DEFAULT_TABLE table=default.Table_085fd4536b524eb9aacb353d95ffa166 [id=c0797cd687f2480087ce8379df004967]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.901611 11107 rpcz_store.cc:276] 0809 01:45:43.475652 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.856505 (+380853us) service_pool.cc:224] Handling call
0809 01:45:43.980318 (+123813us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.980538 (+ 220us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.253025 (+272487us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.421376 (+168351us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.898518 (+477142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:44.912274 11119 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 166) took 1438 ms. Trace:
I20260809 01:45:44.912598 11119 rpcz_store.cc:276] 0809 01:45:43.474267 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.832036 (+357769us) service_pool.cc:224] Handling call
0809 01:45:43.833454 (+ 1418us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.931328 (+ 97874us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.416133 (+484805us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.752317 (+336184us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.912222 (+159905us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:44.913244 11107 tablet_service.cc:1511] Processing CreateTablet for tablet 058dfc9b37f04fd8a72981b4290390b4 (DEFAULT_TABLE table=default.Table_a97d9950868e4ee0b869484bdde6827f [id=0cff66be49af4f69b1ddb65d69af11c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:44.913779 11129 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 158) took 1619 ms. Trace:
I20260809 01:45:44.914130 11129 rpcz_store.cc:276] 0809 01:45:43.294557 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.638137 (+343580us) service_pool.cc:224] Handling call
0809 01:45:43.672447 (+ 34310us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.689571 (+ 17124us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.745915 (+1056344us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.757428 (+ 11513us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.913733 (+156305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:44.915081 11129 tablet_service.cc:1558] Processing DeleteTablet for tablet 403cb09c1786491ab0d40502038bf9c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:43 UTC) from {username='slave'} at 127.0.0.1:56566
W20260809 01:45:44.916236 10804 leader_election.cc:343] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.916520 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3bf43978dc94c2b9a7f3c18a4677c40" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:44.916589 10804 leader_election.cc:304] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.917146 12211 raft_consensus.cc:493] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:44.917569 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31874b7e70a847a4885bd09da40a6902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.917501 12211 raft_consensus.cc:515] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.917867 11154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38bd344108649ed8e1687f82154000b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:44.918984 11119 tablet_service.cc:1558] Processing DeleteTablet for tablet c3bf43978dc94c2b9a7f3c18a4677c40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:43 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:44.919246 12211 leader_election.cc:290] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:44.919639 12211 raft_consensus.cc:2749] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:44.920759 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da86b5e4a27544c59ae6bb387440d246" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:44.921200 11158 raft_consensus.cc:3060] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:44.921782 11112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3b02bd8c93b43bf90ff0192a9541aae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.925215 11107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 058dfc9b37f04fd8a72981b4290390b4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:44.926661 10804 leader_election.cc:343] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.926991 10804 leader_election.cc:304] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.927542 12211 raft_consensus.cc:2749] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:44.928521 10804 leader_election.cc:343] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.928835 10804 leader_election.cc:304] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.931280 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3bf43978dc94c2b9a7f3c18a4677c40" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:44.932426 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2692a931500443d298e25a2b362b22a1" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:44.933794 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9465f8bb39d64e1cab8646282cb65c75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.937824 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 5050a590fe0240ac8c9f9191ec2475a1 (table default.Table_fe1acd943c5c42c9b09a0a5845cbee68 [id=487f10bf79a944168eece2b1e8a2daf0]) successfully deleted
I20260809 01:45:44.938138 12211 raft_consensus.cc:2749] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:44.938767 11106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c704d148da8d406aa824253a51b850c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.939693 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 183) took 2507 ms. Trace:
I20260809 01:45:44.940061 11949 rpcz_store.cc:276] 0809 01:45:42.432472 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.432578 (+ 106us) service_pool.cc:224] Handling call
0809 01:45:44.819922 (+2387344us) ts_tablet_manager.cc:1151] Deleting tablet b2726cb09f734ed18bb9379f94f3f22a
0809 01:45:44.819958 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:44.878892 (+ 58934us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.903908 (+ 25016us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.939636 (+ 35728us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":675,"tablet-delete.queue_time_us":2344959}
I20260809 01:45:44.941289 11949 tablet_replica.cc:333] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:44.942178 11949 raft_consensus.cc:2243] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:44.942633 11949 raft_consensus.cc:2272] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:44.944365 11949 ts_tablet_manager.cc:1916] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:44.946092 10804 leader_election.cc:343] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:44.947213 10804 leader_election.cc:343] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.947551 10804 leader_election.cc:304] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.948312 12339 raft_consensus.cc:2749] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:44.949649 12276 consensus_queue.cc:1048] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:44.950403 11553 tablet_bootstrap.cc:492] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:44.950760 11553 ts_tablet_manager.cc:1403] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.077s user 0.007s sys 0.002s
I20260809 01:45:44.952878 11553 raft_consensus.cc:359] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.953495 11553 raft_consensus.cc:385] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:44.953763 11553 raft_consensus.cc:740] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:44.955030 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet b2726cb09f734ed18bb9379f94f3f22a (table default.Table_03cfb94f08614dc5bcdaf9eb182adc1b [id=d23f238a284a47f2be2faf340b853440]) successfully deleted
I20260809 01:45:44.955348 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2692a931500443d298e25a2b362b22a1" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:44.957837 11289 consensus_peers.cc:597] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 12fb852b8ea8478cb160a265459865c4. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:44.961659 12339 raft_consensus.cc:493] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:44.962167 12339 raft_consensus.cc:515] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.964203 12339 leader_election.cc:290] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:44.965691 10804 leader_election.cc:343] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.965795 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3684fa56f9549f492a96a5c765eda78" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:44.966281 11553 consensus_queue.cc:260] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:44.968153 11553 ts_tablet_manager.cc:1434] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260809 01:45:44.960800 11334 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 193) took 1955 ms. Trace:
I20260809 01:45:44.968997 11334 rpcz_store.cc:276] 0809 01:45:43.005797 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.940168 (+934371us) service_pool.cc:224] Handling call
0809 01:45:43.976234 (+ 36066us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.976378 (+ 144us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.756092 (+779714us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.960341 (+204249us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:44.960742 (+ 401us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260809 01:45:44.971827 10804 leader_election.cc:343] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.973922 12279 tablet.cc:1724] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:44.974675 11553 tablet_bootstrap.cc:492] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
W20260809 01:45:44.957274 10805 leader_election.cc:343] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:44.989014 10805 leader_election.cc:304] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:44.989941 12339 raft_consensus.cc:2749] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:44.990041 12300 raft_consensus.cc:493] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:44.990638 12300 raft_consensus.cc:515] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:44.991201 10592 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:37956 (request call id 252) took 1327 ms. Trace:
I20260809 01:45:44.994123 12300 leader_election.cc:290] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:44.991613 10592 rpcz_store.cc:276] 0809 01:45:43.662820 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.682549 (+ 19729us) service_pool.cc:224] Handling call
0809 01:45:43.684850 (+ 2301us) catalog_manager.cc:2127] Acquired catalog manager lock
0809 01:45:43.686492 (+ 1642us) catalog_manager.cc:2173] Created new table and tablet info
0809 01:45:44.989018 (+1302526us) catalog_manager.cc:2233] Wrote table and tablets to system table
0809 01:45:44.989826 (+ 808us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0809 01:45:44.990372 (+ 546us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:44.332328 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:44.341211 (+ 8883us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:44.341246 (+ 35us) write_op.cc:435] Acquired schema lock
0809 01:45:44.341265 (+ 19us) tablet.cc:662] Decoding operations
0809 01:45:44.638024 (+296759us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:44.638088 (+ 64us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:44.638126 (+ 38us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:44.638415 (+ 289us) tablet.cc:701] Row locks acquired
0809 01:45:44.638435 (+ 20us) write_op.cc:260] PREPARE: finished
0809 01:45:44.638576 (+ 141us) write_op.cc:270] Start()
0809 01:45:44.638678 (+ 102us) write_op.cc:276] Timestamp: P: 1786239944638498 usec, L: 0
0809 01:45:44.638700 (+ 22us) op_driver.cc:348] REPLICATION: starting
0809 01:45:44.819732 (+181032us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b5400030b18. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c75c9964 00007fa2c67dfd19 00007fa2c679ba01 00007fa2c679b6da 00007fa2c681fd45 00007fa2c68201e9 00007fa2c6caff61 00007fa2c6caf738 00007fa2c6cb43d1 00007fa2c6cb4389 00007fa2c6cb4319
0809 01:45:44.820066 (+ 334us) log.cc:844] Serialized 433 byte log entry
0809 01:45:44.820556 (+ 490us) spinlock_profiling.cc:243] Waited 40.4 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678d951 00007fa2c6820208 00007fa2c6caff61 00007fa2c6caf738 00007fa2c6cb43d1 00007fa2c6cb4389 00007fa2c6cb4319 00007fa2c6cb42e1 00007fa2c6cb350d 00000000005bd8c4 00000000005bd6f9
0809 01:45:44.837360 (+ 16804us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b5800051bd0. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c677e81c 00007fa2c6785133 00007fa2c67850b9 00007fa2c6785049 00007fa2c6785011 00007fa2c67841dd 00000000005bd8c4 00000000005bd6f9 00007fa2bf6bedb5 00007fa2bf6c1121
0809 01:45:44.853964 (+ 16604us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b58000527d0. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c677e81c 00007fa2c6785133 00007fa2c67850b9 00007fa2c6785049 00007fa2c6785011 00007fa2c67841dd 00000000005bd8c4 00000000005bd6f9 00007fa2bf6bedb5 00007fa2bf6c1121
0809 01:45:44.982976 (+129012us) op_driver.cc:464] REPLICATION: finished
0809 01:45:44.983929 (+ 953us) write_op.cc:301] APPLY: starting
0809 01:45:44.984007 (+ 78us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:44.984139 (+ 132us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:44.984223 (+ 84us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:44.985551 (+ 1328us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:44.986012 (+ 461us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 01:45:44.986126 (+ 114us) write_op.cc:312] APPLY: finished
0809 01:45:44.986760 (+ 634us) log.cc:844] Serialized 29 byte log entry
0809 01:45:44.987387 (+ 627us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:44.987532 (+ 145us) write_op.cc:454] Released schema lock
0809 01:45:44.987686 (+ 154us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":711,"apply.run_cpu_time_us":4240,"apply.run_wall_time_us":9950,"mutex_wait_us":390155,"num_ops":2,"prepare.queue_time_us":580336,"prepare.run_cpu_time_us":14950,"prepare.run_wall_time_us":572209,"raft.queue_time_us":15971,"raft.run_cpu_time_us":1542,"raft.run_wall_time_us":31895,"replication_time_us":344153,"spinlock_wait_cycles":481883648,"thread_start_us":81465,"threads_started":1}]]}
I20260809 01:45:44.997543 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e360e79820d148f4a6e9fdeb2c19da54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:44.998701 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_218ebdf2aa774da6b7b15e22e0d703df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:45.004513 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 140) took 3939 ms. Trace:
W20260809 01:45:45.001302 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_218ebdf2aa774da6b7b15e22e0d703df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:45.007187 11553 tablet_bootstrap.cc:654] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:45.009449 11113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3938bfd21b294fa6a25f0d8efc1ca29a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.013669 12291 raft_consensus.cc:493] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f60c9ac9414403d9ea5a97b103c2316)
I20260809 01:45:45.013913 11120 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 176) took 1370 ms. Trace:
I20260809 01:45:45.014263 11120 rpcz_store.cc:276] 0809 01:45:43.643869 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.389967 (+746098us) service_pool.cc:224] Handling call
0809 01:45:44.391316 (+ 1349us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.391389 (+ 73us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.767322 (+375933us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.778771 (+ 11449us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.013848 (+235077us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:45.014359 12291 raft_consensus.cc:515] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:45.004995 12222 rpcz_store.cc:276] 0809 01:45:41.166668 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.223874 (+ 57206us) service_pool.cc:224] Handling call
0809 01:45:41.600436 (+376562us) ts_tablet_manager.cc:1151] Deleting tablet 737d7047a0c74a03b3203112987c0cb2
0809 01:45:41.600476 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:43.839025 (+2238549us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.267814 (+428789us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.004383 (+736569us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4788,"tablet-delete.queue_time_us":375947,"thread_start_us":333690,"threads_started":1}
I20260809 01:45:45.018328 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd3662c307ba4ff78b8f3165b23acb1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.020550 11949 ts_tablet_manager.cc:1929] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:45.021003 11949 log.cc:1199] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/c16897857df24c6da5744a99d5f687c0
I20260809 01:45:45.021790 11949 ts_tablet_manager.cc:1950] T c16897857df24c6da5744a99d5f687c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:45.021495 12291 leader_election.cc:290] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:45.022258 11114 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 174) took 1380 ms. Trace:
I20260809 01:45:45.024317 11111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.024520 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 182) took 2592 ms. Trace:
I20260809 01:45:45.026360 11115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9a0fc14dd684ccaa19d51261b9c610e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.028327 11553 tablet_bootstrap.cc:492] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:45.028708 11553 ts_tablet_manager.cc:1403] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.060s user 0.005s sys 0.008s
I20260809 01:45:45.032109 11553 raft_consensus.cc:359] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:45.032922 11553 raft_consensus.cc:385] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:45.033170 11553 raft_consensus.cc:740] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
W20260809 01:45:45.036140 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 78888.7us
I20260809 01:45:45.038926 11366 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 190) took 2034 ms. Trace:
W20260809 01:45:45.041843 10805 leader_election.cc:343] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:45.042255 10805 leader_election.cc:304] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:45.045825 11366 rpcz_store.cc:276] 0809 01:45:43.004746 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.570307 (+565561us) service_pool.cc:224] Handling call
0809 01:45:44.050162 (+479855us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f193d2 00007f0eeae5dfb9 00007f0eea09f640 00007f0eea184641 00007f0eea1618d3 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82 00007f0ee869cab1
0809 01:45:44.376263 (+326101us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f1984c 00007f0ee1f1940b 00007f0eeae5dfb9 00007f0eea09f640 00007f0eea184641 00007f0eea1618d3 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82 00007f0ee869cab1
0809 01:45:44.427446 (+ 51183us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.427552 (+ 106us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.754128 (+326576us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.028861 (+274733us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.038862 (+ 10001us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":648388864}
I20260809 01:45:45.049597 11361 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 204) took 1725 ms. Trace:
I20260809 01:45:45.041872 11553 consensus_queue.cc:260] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:45.061931 11553 ts_tablet_manager.cc:1434] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.033s user 0.005s sys 0.004s
I20260809 01:45:45.062943 11553 tablet_bootstrap.cc:492] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:45.024966 11949 rpcz_store.cc:276] 0809 01:45:42.431664 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.431784 (+ 120us) service_pool.cc:224] Handling call
0809 01:45:44.941135 (+2509351us) ts_tablet_manager.cc:1151] Deleting tablet c16897857df24c6da5744a99d5f687c0
0809 01:45:44.941183 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:45.002694 (+ 61511us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.020382 (+ 17688us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.024432 (+ 4050us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2459981}
I20260809 01:45:45.066120 11949 tablet_replica.cc:333] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:45.066444 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet c16897857df24c6da5744a99d5f687c0 (table default.Table_6d4e0b3afc864450bdf1a8fb35f7dc75 [id=9b307bc98a124b07a2f46fafb20ce1f0]) successfully deleted
I20260809 01:45:45.066839 11949 raft_consensus.cc:2243] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:45.067234 11949 raft_consensus.cc:2272] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:45.069212 11949 ts_tablet_manager.cc:1916] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:45.074291 10592 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet d0091542071441e5bad0c2dca819f37f (table default.Table_fb9b15123c274d7db877eac3e7906edc [id=266a0465cb4544eba6f3e9df1eacf724]). Expected version 2 got 0
I20260809 01:45:45.059932 11361 rpcz_store.cc:276] 0809 01:45:43.324160 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.081363 (+757203us) service_pool.cc:224] Handling call
0809 01:45:44.379586 (+298223us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f193d2 00007f0eeae5dfb9 00007f0eea09f640 00007f0eea184641 00007f0eea1618d3 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82 00007f0ee869cab1
0809 01:45:44.426374 (+ 46788us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.426482 (+ 108us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.761760 (+335278us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.049118 (+287358us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.049510 (+ 392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352021376}
I20260809 01:45:45.076754 10592 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_afe1f3910a684d509d8eb8112b74cb30" } modify_external_catalogs: true
I20260809 01:45:45.081962 12339 raft_consensus.cc:2749] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:45.083211 11337 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 199) took 1815 ms. Trace:
I20260809 01:45:45.084849 11553 tablet_bootstrap.cc:654] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:45.087266 11123 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 175) took 1444 ms. Trace:
I20260809 01:45:45.083467 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3684fa56f9549f492a96a5c765eda78" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:45.091009 11337 rpcz_store.cc:276] 0809 01:45:43.267536 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.961061 (+693525us) service_pool.cc:224] Handling call
0809 01:45:44.374067 (+413006us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f193d2 00007f0eeae5dfb9 00007f0eea09f640 00007f0eea184641 00007f0eea1618d3 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82 00007f0ee869cab1
0809 01:45:44.427237 (+ 53170us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.427745 (+ 508us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.890219 (+462474us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.082748 (+192529us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.083154 (+ 406us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":600244096}
I20260809 01:45:45.077337 10592 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_afe1f3910a684d509d8eb8112b74cb30" } modify_external_catalogs: true
I20260809 01:45:45.102892 11554 tablet_bootstrap.cc:492] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:45.103482 11554 ts_tablet_manager.cc:1403] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.392s user 0.011s sys 0.011s
I20260809 01:45:45.105979 11554 raft_consensus.cc:359] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:45.106719 11554 raft_consensus.cc:385] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:45.107004 11554 raft_consensus.cc:740] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:45.107586 11554 consensus_queue.cc:260] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:45.111006 11063 leader_election.cc:343] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 2aac67e7819f4a70b1050b558868f27f
W20260809 01:45:45.111436 11063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 79176us
I20260809 01:45:45.118983 11110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e2f77a70794ff7811b5f09d885904b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.022858 11114 rpcz_store.cc:276] 0809 01:45:43.642446 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.386588 (+744142us) service_pool.cc:224] Handling call
0809 01:45:44.388835 (+ 2247us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.388966 (+ 131us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.784781 (+395815us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.791227 (+ 6446us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.022169 (+230942us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:45.130726 12343 raft_consensus.cc:493] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 629f002048f64491afdd57f72a466e59)
I20260809 01:45:45.131333 12343 raft_consensus.cc:515] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:45.136307 12339 raft_consensus.cc:493] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:45.136811 12339 raft_consensus.cc:515] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:45.139005 12339 leader_election.cc:290] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:45.140408 10588 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:37956 (request call id 257) took 1248 ms. Trace:
I20260809 01:45:45.140746 10588 rpcz_store.cc:276] 0809 01:45:43.892038 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.893500 (+ 1462us) service_pool.cc:224] Handling call
0809 01:45:43.896163 (+ 2663us) catalog_manager.cc:2127] Acquired catalog manager lock
0809 01:45:43.907975 (+ 11812us) catalog_manager.cc:2173] Created new table and tablet info
0809 01:45:45.139603 (+1231628us) catalog_manager.cc:2233] Wrote table and tablets to system table
0809 01:45:45.139929 (+ 326us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0809 01:45:45.140243 (+ 314us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:44.904898 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:44.905413 (+ 515us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:44.905448 (+ 35us) write_op.cc:435] Acquired schema lock
0809 01:45:44.905482 (+ 34us) tablet.cc:662] Decoding operations
0809 01:45:44.905926 (+ 444us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:44.906004 (+ 78us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:44.906044 (+ 40us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:44.906310 (+ 266us) tablet.cc:701] Row locks acquired
0809 01:45:44.906328 (+ 18us) write_op.cc:260] PREPARE: finished
0809 01:45:44.906456 (+ 128us) write_op.cc:270] Start()
0809 01:45:44.906553 (+ 97us) write_op.cc:276] Timestamp: P: 1786239944906360 usec, L: 0
0809 01:45:44.906581 (+ 28us) op_driver.cc:348] REPLICATION: starting
0809 01:45:44.907321 (+ 740us) log.cc:844] Serialized 433 byte log entry
0809 01:45:45.135354 (+228033us) op_driver.cc:464] REPLICATION: finished
0809 01:45:45.136053 (+ 699us) write_op.cc:301] APPLY: starting
0809 01:45:45.136113 (+ 60us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:45.136209 (+ 96us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:45.136292 (+ 83us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:45.136675 (+ 383us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:45.136814 (+ 139us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 01:45:45.136853 (+ 39us) write_op.cc:312] APPLY: finished
0809 01:45:45.137019 (+ 166us) log.cc:844] Serialized 29 byte log entry
0809 01:45:45.137163 (+ 144us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:45.137229 (+ 66us) write_op.cc:454] Released schema lock
0809 01:45:45.137257 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:45.137498 (+ 241us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:45.137540 (+ 42us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":545,"apply.run_cpu_time_us":1408,"apply.run_wall_time_us":1408,"mutex_wait_us":265753,"num_ops":2,"prepare.queue_time_us":995329,"prepare.run_cpu_time_us":2708,"prepare.run_wall_time_us":28879,"replication_time_us":228668}]]}
W20260809 01:45:45.144609 10596 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4aa1429f21024739b06bec88c3cb24d9 has no consensus state
I20260809 01:45:45.145076 10596 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:37956 (request call id 258) took 1240 ms. Trace:
I20260809 01:45:45.087597 11123 rpcz_store.cc:276] 0809 01:45:43.643236 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.392020 (+748784us) service_pool.cc:224] Handling call
0809 01:45:44.393301 (+ 1281us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.393378 (+ 77us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.757996 (+364618us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.808418 (+ 50422us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.087212 (+278794us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:45.145328 10596 rpcz_store.cc:276] 0809 01:45:43.904940 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.906905 (+ 1965us) service_pool.cc:224] Handling call
0809 01:45:43.907924 (+ 1019us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:43.908652 (+ 728us) catalog_manager.cc:3751] Apply alter schema
0809 01:45:43.909204 (+ 552us) catalog_manager.cc:3783] Acquired catalog manager lock
0809 01:45:43.909395 (+ 191us) catalog_manager.cc:3941] Updating metadata on disk
0809 01:45:45.143967 (+1234572us) catalog_manager.cc:3972] Committing alterations to in-memory state
0809 01:45:45.145020 (+ 1053us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:44.933880 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:44.934295 (+ 415us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:44.934325 (+ 30us) write_op.cc:435] Acquired schema lock
0809 01:45:44.934337 (+ 12us) tablet.cc:662] Decoding operations
0809 01:45:44.934599 (+ 262us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:44.934632 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:44.934654 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:44.934793 (+ 139us) tablet.cc:701] Row locks acquired
0809 01:45:44.934806 (+ 13us) write_op.cc:260] PREPARE: finished
0809 01:45:44.934917 (+ 111us) write_op.cc:270] Start()
0809 01:45:44.934992 (+ 75us) write_op.cc:276] Timestamp: P: 1786239944934813 usec, L: 0
0809 01:45:44.935006 (+ 14us) op_driver.cc:348] REPLICATION: starting
0809 01:45:44.935486 (+ 480us) log.cc:844] Serialized 405 byte log entry
0809 01:45:45.135580 (+200094us) op_driver.cc:464] REPLICATION: finished
0809 01:45:45.137765 (+ 2185us) write_op.cc:301] APPLY: starting
0809 01:45:45.137813 (+ 48us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:45.137890 (+ 77us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:45.137972 (+ 82us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:45.138113 (+ 141us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:45.138222 (+ 109us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:45.138259 (+ 37us) write_op.cc:312] APPLY: finished
0809 01:45:45.138395 (+ 136us) log.cc:844] Serialized 23 byte log entry
0809 01:45:45.138499 (+ 104us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:45.138543 (+ 44us) write_op.cc:454] Released schema lock
0809 01:45:45.138572 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2065,"apply.run_cpu_time_us":963,"apply.run_wall_time_us":960,"mutex_wait_us":254905,"num_ops":1,"prepare.queue_time_us":1013426,"prepare.run_cpu_time_us":1811,"prepare.run_wall_time_us":2603,"replication_time_us":200502,"threads_started":1}]]}
I20260809 01:45:45.156313 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09a73ee8e8ec496a984efcab27715506" candidate_term: 3 candidate_status { last_received { term: 2 index: 292 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:45:45.158018 11126 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 156) took 2888 ms. Trace:
I20260809 01:45:45.158378 12343 leader_election.cc:290] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:45.158417 11126 rpcz_store.cc:276] 0809 01:45:42.280582 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.753762 (+473180us) service_pool.cc:224] Handling call
0809 01:45:42.755021 (+ 1259us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:43.602326 (+847305us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.288429 (+686103us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:44.770238 (+481809us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.157963 (+387725us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45858}
W20260809 01:45:45.163281 10320 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
I20260809 01:45:45.173454 11553 tablet_bootstrap.cc:492] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:45.171908 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761370ec8b164fd184a2e091030f610b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:45.176919 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 737d7047a0c74a03b3203112987c0cb2 (table default.Table_cf7bd2b9aec04c73b8ed72debcab795d [id=ccd0ea07da604f859047f2eca85fc566]) successfully deleted
W20260809 01:45:45.179140 10320 leader_election.cc:336] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063): Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
I20260809 01:45:45.179710 10320 leader_election.cc:304] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09a73ee8e8ec496a984efcab27715506; no voters: 629f002048f64491afdd57f72a466e59, 71bc72deb1d2402cabd817dfe3b27e88
I20260809 01:45:45.180642 11553 ts_tablet_manager.cc:1403] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.118s user 0.011s sys 0.001s
I20260809 01:45:45.183228 12367 tablet.cc:1724] T d0091542071441e5bad0c2dca819f37f P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:45.194576 11553 raft_consensus.cc:359] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:45.198213 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 141) took 3972 ms. Trace:
I20260809 01:45:45.205754 10588 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_c7666e50daa54efdab20517273b73423"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:45.208012 10588 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c7666e50daa54efdab20517273b73423 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:45.198753 12222 rpcz_store.cc:276] 0809 01:45:41.225622 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.226769 (+ 1147us) service_pool.cc:224] Handling call
0809 01:45:45.197766 (+3970997us) ts_tablet_manager.cc:1151] Deleting tablet c795f81d2bc248eeb39b64d6aaa0370c
0809 01:45:45.197868 (+ 102us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:45.198136 (+ 268us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3968923}
I20260809 01:45:45.233733 12222 tablet_replica.cc:333] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:45.238370 10596 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet d0091542071441e5bad0c2dca819f37f (table default.Table_fb9b15123c274d7db877eac3e7906edc [id=266a0465cb4544eba6f3e9df1eacf724]). Expected version 2 got 0
I20260809 01:45:45.233191 10595 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet e3684fa56f9549f492a96a5c765eda78 (table default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]). Expected version 1 got 0
W20260809 01:45:45.246276 10595 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 35) took 925 ms (client timeout 1000 ms). Trace:
W20260809 01:45:45.246784 10595 rpcz_store.cc:269] 0809 01:45:44.321085 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.355897 (+ 34812us) service_pool.cc:224] Handling call
0809 01:45:45.246107 (+890210us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":4}
W20260809 01:45:45.257846 10576 catalog_manager.cc:5017] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet c795f81d2bc248eeb39b64d6aaa0370c because the tablet was not found. No further retry: Not found: Tablet not found: c795f81d2bc248eeb39b64d6aaa0370c
I20260809 01:45:45.272511 11117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 170) took 1797 ms. Trace:
W20260809 01:45:45.204916 10804 leader_election.cc:343] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:45.289836 10804 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 84950.5us
I20260809 01:45:45.291368 11127 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 187) took 1414 ms. Trace:
I20260809 01:45:45.291797 11127 rpcz_store.cc:276] 0809 01:45:43.877187 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.900119 (+1022932us) service_pool.cc:224] Handling call
0809 01:45:44.901200 (+ 1081us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.901264 (+ 64us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.973373 (+ 72109us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.133667 (+160294us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.291261 (+157594us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:45.292462 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 66cc1253dee24f5d858d521ddbbfb620 (DEFAULT_TABLE table=default.Table_afe1f3910a684d509d8eb8112b74cb30 [id=110d6d2cbdb2458bb4785bf0ab835236]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.293769 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66cc1253dee24f5d858d521ddbbfb620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.295342 11126 tablet_service.cc:1511] Processing CreateTablet for tablet 66cc1253dee24f5d858d521ddbbfb620 (DEFAULT_TABLE table=default.Table_afe1f3910a684d509d8eb8112b74cb30 [id=110d6d2cbdb2458bb4785bf0ab835236]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.296557 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66cc1253dee24f5d858d521ddbbfb620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.296991 10852 tablet_service.cc:1558] Processing DeleteTablet for tablet b2726cb09f734ed18bb9379f94f3f22a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:41 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:45.298060 12339 raft_consensus.cc:493] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:45.298583 10858 tablet_service.cc:1511] Processing CreateTablet for tablet e5af40994c204c4f82e36a53e00f2dfe (DEFAULT_TABLE table=default.Table_52e499c473a24748b65fa3b389cf857a [id=e995a84f2635461ca81f05d2ece827c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.298672 12339 raft_consensus.cc:515] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:45.299892 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5af40994c204c4f82e36a53e00f2dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.301437 12339 leader_election.cc:290] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:45.310184 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84f1943c1a34e999fc7c12a5847cdb6" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:45.312207 10804 leader_election.cc:343] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:45.321708 11553 raft_consensus.cc:385] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:45.322039 11553 raft_consensus.cc:740] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:45.322651 11949 ts_tablet_manager.cc:1929] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:45.323326 11949 log.cc:1199] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/197b5043f6d348f89853c073b256daa4
I20260809 01:45:45.322875 11553 consensus_queue.cc:260] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:45.324092 11949 ts_tablet_manager.cc:1950] T 197b5043f6d348f89853c073b256daa4 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:45.325987 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 190) took 2885 ms. Trace:
I20260809 01:45:45.326277 11949 rpcz_store.cc:276] 0809 01:45:42.440344 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.442868 (+ 2524us) service_pool.cc:224] Handling call
0809 01:45:45.065920 (+2623052us) ts_tablet_manager.cc:1151] Deleting tablet 197b5043f6d348f89853c073b256daa4
0809 01:45:45.065979 (+ 59us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:45.077034 (+ 11055us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.322521 (+245487us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.325932 (+ 3411us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2565113}
I20260809 01:45:45.330272 11125 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 179) took 1641 ms. Trace:
I20260809 01:45:45.330638 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 197b5043f6d348f89853c073b256daa4 (table default.Table_5f5f7b17133f4f4d9cfcac1aa0d270fa [id=12a77217eec442c6a9199a418d9d5bb8]) successfully deleted
W20260809 01:45:45.331039 12336 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s user 0.061s sys 0.079s
W20260809 01:45:45.331432 12336 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s user 0.061s sys 0.080s
I20260809 01:45:45.334118 11124 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 178) took 1689 ms. Trace:
I20260809 01:45:45.334755 11949 tablet_replica.cc:333] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:45.335433 11949 raft_consensus.cc:2243] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:45.335799 11949 raft_consensus.cc:2272] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:45.337221 11949 ts_tablet_manager.cc:1916] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:45.330603 11125 rpcz_store.cc:276] 0809 01:45:43.706622 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.441956 (+735334us) service_pool.cc:224] Handling call
0809 01:45:44.897662 (+455706us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.897741 (+ 79us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.978137 (+ 80396us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.320229 (+342092us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.330217 (+ 9988us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9585}
I20260809 01:45:45.334470 11124 rpcz_store.cc:276] 0809 01:45:43.645235 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.440951 (+795716us) service_pool.cc:224] Handling call
0809 01:45:44.997026 (+556075us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.997114 (+ 88us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:45.127669 (+130555us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.333699 (+206030us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.334069 (+ 370us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11129}
I20260809 01:45:45.022480 11334 tablet_service.cc:1558] Processing DeleteTablet for tablet 7501d985958746bea58e0ff752d7fed8 with delete_type TABLET_DATA_DELETED (replaced by tablet b2726cb09f734ed18bb9379f94f3f22a) from {username='slave'} at 127.0.0.1:53476
W20260809 01:45:45.345050 10805 leader_election.cc:343] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:45.345484 10805 leader_election.cc:304] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:45.346160 12339 raft_consensus.cc:2749] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:45.096745 11351 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 201) took 1773 ms. Trace:
I20260809 01:45:45.142016 11353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 203) took 1818 ms. Trace:
I20260809 01:45:45.368300 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_62a39e76a73049dea65f8948c5513aa1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:45.372334 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_62a39e76a73049dea65f8948c5513aa1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:45.168828 11554 ts_tablet_manager.cc:1434] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.065s user 0.005s sys 0.000s
I20260809 01:45:45.376849 11554 tablet_bootstrap.cc:492] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:45.135922 11951 ts_tablet_manager.cc:1929] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:45.377691 11553 ts_tablet_manager.cc:1434] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.197s user 0.001s sys 0.005s
I20260809 01:45:45.377948 11951 log.cc:1199] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/ac6eded297214b58a5d8129a60ca1120
I20260809 01:45:45.379041 11951 ts_tablet_manager.cc:1950] T ac6eded297214b58a5d8129a60ca1120 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:45.381275 11553 tablet_bootstrap.cc:492] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:45.109084 12361 tablet.cc:1724] T d0091542071441e5bad0c2dca819f37f P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:45.386804 10852 tablet_service.cc:1511] Processing CreateTablet for tablet 34616fd5895945b3aa896d5a8ee87249 (DEFAULT_TABLE table=default.Table_0ebfe415d8ef4946b8971a58797d97c6 [id=946eac74789f4001b253e66f9ff48fe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.387861 10852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34616fd5895945b3aa896d5a8ee87249. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.391551 12343 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 01:45:45.347404 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761370ec8b164fd184a2e091030f610b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:45.398413 10589 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cef88ffe9e95437da81793795742def3 consensus state has no leader
I20260809 01:45:45.350479 11347 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 194) took 2344 ms. Trace:
I20260809 01:45:45.399483 11347 rpcz_store.cc:276] 0809 01:45:43.006183 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.952652 (+946469us) service_pool.cc:224] Handling call
0809 01:45:44.381008 (+428356us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f193d2 00007f0eeae5dfb9 00007f0eea09f640 00007f0eea184641 00007f0eea1618d3 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82 00007f0ee869cab1
0809 01:45:44.431308 (+ 50300us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.431571 (+ 263us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.889031 (+457460us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.094436 (+205405us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.350218 (+255782us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":955154816}
I20260809 01:45:45.400719 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 112) took 7163 ms. Trace:
W20260809 01:45:45.401315 10805 leader_election.cc:343] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:45.401116 11951 rpcz_store.cc:276] 0809 01:45:38.237471 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.237627 (+ 156us) service_pool.cc:224] Handling call
0809 01:45:44.652082 (+6414455us) ts_tablet_manager.cc:1151] Deleting tablet ac6eded297214b58a5d8129a60ca1120
0809 01:45:44.652123 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:45.035844 (+383721us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.135782 (+ 99938us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.400617 (+264835us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6390315}
I20260809 01:45:45.401710 10805 leader_election.cc:304] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:45.351115 11353 rpcz_store.cc:276] 0809 01:45:43.323927 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.991836 (+667909us) service_pool.cc:224] Handling call
0809 01:45:44.373215 (+381379us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f193d2 00007f0eeae5dfb9 00007f0eea09f640 00007f0eea184641 00007f0eea1618d3 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82 00007f0ee869cab1
0809 01:45:44.436975 (+ 63760us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.437313 (+ 338us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.966654 (+529341us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.099614 (+132960us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.141951 (+ 42337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":545274752}
I20260809 01:45:45.402537 11951 tablet_replica.cc:333] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:45.402887 12339 raft_consensus.cc:2749] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:45.403532 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet ac6eded297214b58a5d8129a60ca1120 (table default.Table_c92dfb9ed1d342f197b158bef20fc0ac [id=96595f8bb6b84744bf7fc74f074a51ba]) successfully deleted
I20260809 01:45:45.353380 11337 tablet_service.cc:1511] Processing CreateTablet for tablet e5af40994c204c4f82e36a53e00f2dfe (DEFAULT_TABLE table=default.Table_52e499c473a24748b65fa3b389cf857a [id=e995a84f2635461ca81f05d2ece827c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.405747 11337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5af40994c204c4f82e36a53e00f2dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.406895 11554 tablet_bootstrap.cc:654] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:45.354183 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84f1943c1a34e999fc7c12a5847cdb6" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:45.272966 11117 rpcz_store.cc:276] 0809 01:45:43.475312 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.895938 (+420626us) service_pool.cc:224] Handling call
0809 01:45:44.263853 (+367915us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.263972 (+ 119us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.877010 (+613038us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.271957 (+394947us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.272435 (+ 478us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:45.355361 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aac67e7819f4a70b1050b558868f27f" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:45.385850 11158 raft_consensus.cc:2468] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:45.412949 11951 raft_consensus.cc:2243] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:45.415221 11127 tablet_service.cc:1511] Processing CreateTablet for tablet e5af40994c204c4f82e36a53e00f2dfe (DEFAULT_TABLE table=default.Table_52e499c473a24748b65fa3b389cf857a [id=e995a84f2635461ca81f05d2ece827c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.416257 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5af40994c204c4f82e36a53e00f2dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.369625 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 34616fd5895945b3aa896d5a8ee87249 (DEFAULT_TABLE table=default.Table_0ebfe415d8ef4946b8971a58797d97c6 [id=946eac74789f4001b253e66f9ff48fe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.416747 11951 pending_rounds.cc:70] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316: Trying to abort 1 pending ops.
I20260809 01:45:45.417251 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34616fd5895945b3aa896d5a8ee87249. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:45.422931 10805 leader_election.cc:343] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:45.423348 10805 leader_election.cc:304] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:45.410811 11116 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 173) took 1933 ms. Trace:
I20260809 01:45:45.429724 11116 rpcz_store.cc:276] 0809 01:45:43.641660 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.263319 (+621659us) service_pool.cc:224] Handling call
0809 01:45:44.264576 (+ 1257us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.264651 (+ 75us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.889148 (+624497us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.301269 (+412121us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.409985 (+108716us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:45.431159 11118 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 188) took 1553 ms. Trace:
I20260809 01:45:45.431495 11118 rpcz_store.cc:276] 0809 01:45:44.128745 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.903267 (+774522us) service_pool.cc:224] Handling call
0809 01:45:44.904494 (+ 1227us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.904578 (+ 84us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.978174 (+ 73596us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.430711 (+452537us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.431102 (+ 391us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:45.433308 12339 raft_consensus.cc:2749] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:45.434587 11124 tablet_service.cc:1558] Processing DeleteTablet for tablet 201329c676ec4b6abbb72b22519bc41d with delete_type TABLET_DATA_DELETED (replaced by tablet 8baa4ae62edf46b5a386661a6826bbab) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:45.434628 11125 tablet_service.cc:1511] Processing CreateTablet for tablet 34616fd5895945b3aa896d5a8ee87249 (DEFAULT_TABLE table=default.Table_0ebfe415d8ef4946b8971a58797d97c6 [id=946eac74789f4001b253e66f9ff48fe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.435609 11125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34616fd5895945b3aa896d5a8ee87249. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.412190 12246 tablet.cc:1724] T d0091542071441e5bad0c2dca819f37f P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:45.417012 11951 pending_rounds.cc:77] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438793105510400 op_type: NO_OP noop_request { }
W20260809 01:45:45.442024 10580 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet cef88ffe9e95437da81793795742def3 (table default.Table_c54e4ecef7c145cc970af6169c5a9326 [id=40cb4403a8a94bd3938983441e9254d4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cef88ffe9e95437da81793795742def3 consensus state has no leader
I20260809 01:45:45.449623 10856 tablet_service.cc:1511] Processing CreateTablet for tablet 778d1582d67d44b9bc25a1db6d819ccb (DEFAULT_TABLE table=default.Table_73c83efa21c2474984a83ffd88538ad2 [id=691bb980619949b4a2c0db94dc5f68e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.450708 10856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778d1582d67d44b9bc25a1db6d819ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.451136 10595 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 3e03033672e947eab8cc143263281133 (table default.Table_79500ae2591d480f93bd129b5e4e5804 [id=24d2e675d969421fba4647380880f606]). Expected version 1 got 0
I20260809 01:45:45.452107 10595 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet a38bd344108649ed8e1687f82154000b (table default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]). Expected version 1 got 0
I20260809 01:45:45.452096 11116 tablet_service.cc:1511] Processing CreateTablet for tablet 778d1582d67d44b9bc25a1db6d819ccb (DEFAULT_TABLE table=default.Table_73c83efa21c2474984a83ffd88538ad2 [id=691bb980619949b4a2c0db94dc5f68e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.413267 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501d985958746bea58e0ff752d7fed8" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:45.461751 12222 raft_consensus.cc:2243] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:45.463097 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778d1582d67d44b9bc25a1db6d819ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.464035 12222 pending_rounds.cc:70] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7: Trying to abort 1 pending ops.
I20260809 01:45:45.464298 12222 pending_rounds.cc:77] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438807881728000 op_type: NO_OP noop_request { }
I20260809 01:45:45.464694 12222 raft_consensus.cc:2889] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:45.465030 12222 raft_consensus.cc:2272] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:45.352252 11334 tablet_service.cc:1511] Processing CreateTablet for tablet 66cc1253dee24f5d858d521ddbbfb620 (DEFAULT_TABLE table=default.Table_afe1f3910a684d509d8eb8112b74cb30 [id=110d6d2cbdb2458bb4785bf0ab835236]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.478744 11951 raft_consensus.cc:2889] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
W20260809 01:45:45.344527 11289 leader_election.cc:343] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 7501d985958746bea58e0ff752d7fed8
I20260809 01:45:45.372172 11346 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 202) took 2048 ms. Trace:
I20260809 01:45:45.481746 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 750aa437ef594587a512a9c9c8278aa8 (DEFAULT_TABLE table=default.Table_5c33dd5878b34a619bf65e590d9aca96 [id=496a5686a0fd4909b87e94557080d890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.482949 12222 ts_tablet_manager.cc:1916] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:45.485024 11551 tablet_bootstrap.cc:654] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:45.479748 11951 raft_consensus.cc:2272] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:45.481381 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 137190us
I20260809 01:45:45.488986 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 750aa437ef594587a512a9c9c8278aa8 (DEFAULT_TABLE table=default.Table_5c33dd5878b34a619bf65e590d9aca96 [id=496a5686a0fd4909b87e94557080d890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.491544 11951 ts_tablet_manager.cc:1916] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:45.491668 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 328c81bb8a534ee885c187f666c0f7ea (DEFAULT_TABLE table=default.Table_b6462a5c7160450f8605d2659d707573 [id=639b70ffde7d4c21938de3ece5324622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.492168 11347 tablet_service.cc:1511] Processing CreateTablet for tablet 750aa437ef594587a512a9c9c8278aa8 (DEFAULT_TABLE table=default.Table_5c33dd5878b34a619bf65e590d9aca96 [id=496a5686a0fd4909b87e94557080d890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.495306 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 750aa437ef594587a512a9c9c8278aa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.496313 11118 tablet_service.cc:1511] Processing CreateTablet for tablet 328c81bb8a534ee885c187f666c0f7ea (DEFAULT_TABLE table=default.Table_b6462a5c7160450f8605d2659d707573 [id=639b70ffde7d4c21938de3ece5324622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.497237 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 328c81bb8a534ee885c187f666c0f7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.497833 11334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66cc1253dee24f5d858d521ddbbfb620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.498437 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 328c81bb8a534ee885c187f666c0f7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.499833 11062 leader_election.cc:304] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:45.500590 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 778d1582d67d44b9bc25a1db6d819ccb (DEFAULT_TABLE table=default.Table_73c83efa21c2474984a83ffd88538ad2 [id=691bb980619949b4a2c0db94dc5f68e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.501806 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778d1582d67d44b9bc25a1db6d819ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.505692 11113 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 172) took 2029 ms. Trace:
I20260809 01:45:45.481648 11346 rpcz_store.cc:276] 0809 01:45:43.323639 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.988384 (+664745us) service_pool.cc:224] Handling call
0809 01:45:44.381926 (+393542us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f193d2 00007f0eeae5dfb9 00007f0eea09f640 00007f0eea184641 00007f0eea1618d3 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82 00007f0ee869cab1
0809 01:45:44.437650 (+ 55724us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.454136 (+ 16486us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.879817 (+425681us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.371391 (+491574us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.372013 (+ 622us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557777152}
I20260809 01:45:45.506043 11113 rpcz_store.cc:276] 0809 01:45:43.475979 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.244509 (+768530us) service_pool.cc:224] Handling call
0809 01:45:45.008628 (+764119us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:45.008826 (+ 198us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:45.278209 (+269383us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.429473 (+151264us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.505623 (+ 76150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7040}
I20260809 01:45:45.350809 11351 rpcz_store.cc:276] 0809 01:45:43.323363 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.973504 (+650141us) service_pool.cc:224] Handling call
0809 01:45:44.380465 (+406961us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f193d2 00007f0eeae5dfb9 00007f0eea09f640 00007f0eea184641 00007f0eea1618d3 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82 00007f0ee869cab1
0809 01:45:44.431942 (+ 51477us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:44.432204 (+ 262us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:44.835651 (+403447us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.096280 (+260629us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.096657 (+ 377us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":575720320}
I20260809 01:45:45.507501 11347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 750aa437ef594587a512a9c9c8278aa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.510360 11283 leader_election.cc:304] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:45.511619 12299 raft_consensus.cc:2804] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
W20260809 01:45:45.512079 11283 leader_election.cc:343] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:45.512112 12299 raft_consensus.cc:697] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:45.512519 11283 leader_election.cc:304] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:45.513023 12299 consensus_queue.cc:237] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:45.522135 12338 raft_consensus.cc:2749] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:45:45.523509 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 750aa437ef594587a512a9c9c8278aa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.527532 12339 raft_consensus.cc:493] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:45.527973 12339 raft_consensus.cc:515] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:45.530354 12339 leader_election.cc:290] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:45.531942 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403cb09c1786491ab0d40502038bf9c0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:45.533233 10804 leader_election.cc:343] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:45.539820 12276 raft_consensus.cc:2749] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:45.479606 11111 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 180) took 1727 ms. Trace:
I20260809 01:45:45.541721 11111 rpcz_store.cc:276] 0809 01:45:43.793990 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.492727 (+698737us) service_pool.cc:224] Handling call
0809 01:45:44.793948 (+301221us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b5800000f98. stack: 00007fe4bc3e2cf3 00007fe4bc3e2bf0 00007fe4bbf90574 0000000000516de4 0000000000516d79 00007fe4bc29d981 00007fe4bc3dd84c 00007fe4bc3dd40b 00007fe4c5321fb9 00007fe4c4563640 00007fe4c4648641 00007fe4c46258d3 00007fe4c45d4210 00007fe4c2b60bf4 00007fe4c2b60b82 00007fe4c2b60ab1
0809 01:45:45.023637 (+229689us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:45.023828 (+ 191us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:45.268500 (+244672us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.432690 (+164190us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.472023 (+ 39333us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":530,"spinlock_wait_cycles":8849280}
I20260809 01:45:45.546324 12222 ts_tablet_manager.cc:1929] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:45.546741 12222 log.cc:1199] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/12fb852b8ea8478cb160a265459865c4
I20260809 01:45:45.547528 12222 ts_tablet_manager.cc:1950] T 12fb852b8ea8478cb160a265459865c4 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:45.548225 11115 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 181) took 1645 ms. Trace:
I20260809 01:45:45.548614 11115 rpcz_store.cc:276] 0809 01:45:43.873711 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.502391 (+628680us) service_pool.cc:224] Handling call
0809 01:45:44.796549 (+294158us) spinlock_profiling.cc:243] Waited 5.39 ms on lock 0x7b5800000f98. stack: 00007fe4bc3e2cf3 00007fe4bc3e2bf0 00007fe4bbf90574 0000000000516de4 0000000000516d79 00007fe4bc29d981 00007fe4bc3dd84c 00007fe4bc3dd40b 00007fe4c5321fb9 00007fe4c4563640 00007fe4c4648641 00007fe4c46258d3 00007fe4c45d4210 00007fe4c2b60bf4 00007fe4c2b60b82 00007fe4c2b60ab1
0809 01:45:45.025609 (+229060us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:45.025785 (+ 176us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:45.282908 (+257123us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.451507 (+168599us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.519384 (+ 67877us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12407040}
I20260809 01:45:45.549868 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 142) took 4323 ms. Trace:
I20260809 01:45:45.550205 12339 raft_consensus.cc:493] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:45.550232 12222 rpcz_store.cc:276] 0809 01:45:41.226054 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.228891 (+ 2837us) service_pool.cc:224] Handling call
0809 01:45:45.233486 (+4004595us) ts_tablet_manager.cc:1151] Deleting tablet 12fb852b8ea8478cb160a265459865c4
0809 01:45:45.233584 (+ 98us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:45.463827 (+230243us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b5400110154. stack: 00007fe4bc3e2cf3 00007fe4bc3e2bf0 00007fe4bbf90574 0000000000516de4 0000000000516d79 00007fe4c34e3951 00007fe4c3582742 00007fe4c39c2d2c 00007fe4c462a41a 00007fe4c465b0e0 00007fe4c464164d 00007fe4c46415f9 00007fe4c4641589 00007fe4c4641551 00007fe4c464077d 00000000005bd8c4
0809 01:45:45.539577 (+ 75750us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.546207 (+ 6630us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.549804 (+ 3597us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521555200,"tablet-delete.queue_time_us":4004058}
I20260809 01:45:45.550801 12339 raft_consensus.cc:515] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:45.551877 12222 tablet_replica.cc:333] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:45.551978 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 12fb852b8ea8478cb160a265459865c4 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]) successfully deleted
I20260809 01:45:45.552637 12222 raft_consensus.cc:2243] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:45.553153 12222 raft_consensus.cc:2272] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:45.549577 10595 catalog_manager.cc:5911] Failed to send AlterTable RPC for tablet a38bd344108649ed8e1687f82154000b (table default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet a38bd344108649ed8e1687f82154000b consensus state has no leader
I20260809 01:45:45.554441 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7a511512184f53921cea081f82aa37" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:45.555722 10804 leader_election.cc:343] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:45.576895 12390 raft_consensus.cc:493] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:45.577335 12390 raft_consensus.cc:515] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:45.578069 10877 tablet_service.cc:1511] Processing CreateTablet for tablet 328c81bb8a534ee885c187f666c0f7ea (DEFAULT_TABLE table=default.Table_b6462a5c7160450f8605d2659d707573 [id=639b70ffde7d4c21938de3ece5324622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.579439 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 328c81bb8a534ee885c187f666c0f7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.579736 12339 leader_election.cc:290] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:45.580680 12390 leader_election.cc:290] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:45.582391 10861 tablet_service.cc:1511] Processing CreateTablet for tablet c11b0394533a485294ace9ee5e99bb19 (DEFAULT_TABLE table=default.Table_ab740dbd782f44eb8f423baeede1de0f [id=92630f3e58394023a931a5e2c40e8591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.583503 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c11b0394533a485294ace9ee5e99bb19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.602855 11115 tablet_service.cc:1511] Processing CreateTablet for tablet c11b0394533a485294ace9ee5e99bb19 (DEFAULT_TABLE table=default.Table_ab740dbd782f44eb8f423baeede1de0f [id=92630f3e58394023a931a5e2c40e8591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.604020 11115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c11b0394533a485294ace9ee5e99bb19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.606463 11110 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 186) took 1729 ms. Trace:
I20260809 01:45:45.607239 12222 ts_tablet_manager.cc:1916] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:45.606937 11111 tablet_service.cc:1511] Processing CreateTablet for tablet 6f771c8051784da18a6de9fcf45e0125 (DEFAULT_TABLE table=default.Table_52926e76c3ad4bbcbc425c32d4473a3c [id=51771c6c664942e4b1df1132b5641ffa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.606819 11110 rpcz_store.cc:276] 0809 01:45:43.876842 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.727413 (+850571us) service_pool.cc:224] Handling call
0809 01:45:45.085874 (+358461us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:45.118586 (+ 32712us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:45.276339 (+157753us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.605842 (+329503us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.606396 (+ 554us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:45.608206 11111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f771c8051784da18a6de9fcf45e0125. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.608387 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389407e7ff464e0d8e84bbdbc4c6081e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:45.611778 12246 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:56566 (request call id 177) took 1967 ms. Trace:
W20260809 01:45:45.613373 10804 leader_election.cc:343] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:45.620518 10852 tablet_service.cc:1511] Processing CreateTablet for tablet 6f771c8051784da18a6de9fcf45e0125 (DEFAULT_TABLE table=default.Table_52926e76c3ad4bbcbc425c32d4473a3c [id=51771c6c664942e4b1df1132b5641ffa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.612115 12246 rpcz_store.cc:276] 0809 01:45:43.644414 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.429798 (+785384us) service_pool.cc:224] Handling call
0809 01:45:45.611702 (+1181904us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:44.508775 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0809 01:45:44.864553 (+355778us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 01:45:44.864585 (+ 32us) alter_schema_op.cc:66] Acquired schema lock
0809 01:45:44.864652 (+ 67us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0809 01:45:44.886301 (+ 21649us) alter_schema_op.cc:128] START. Timestamp: 1786239944884440
0809 01:45:44.886333 (+ 32us) op_driver.cc:348] REPLICATION: starting
0809 01:45:44.886911 (+ 578us) log.cc:844] Serialized 173 byte log entry
0809 01:45:45.332187 (+445276us) op_driver.cc:464] REPLICATION: finished
0809 01:45:45.398538 (+ 66351us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0809 01:45:45.445864 (+ 47326us) log.cc:844] Serialized 5 byte log entry
0809 01:45:45.562610 (+116746us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.611457 (+ 48847us) log.cc:844] Serialized 14 byte log entry
0809 01:45:45.611552 (+ 95us) alter_schema_op.cc:73] Released schema lock
0809 01:45:45.611579 (+ 27us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1734,"prepare.queue_time_us":888,"prepare.run_cpu_time_us":14093,"prepare.run_wall_time_us":415923,"raft.queue_time_us":200,"raft.run_cpu_time_us":802,"raft.run_wall_time_us":822,"replication_time_us":445683,"thread_start_us":37365,"threads_started":1,"wal-append.queue_time_us":15849}]]}
I20260809 01:45:45.622061 10852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f771c8051784da18a6de9fcf45e0125. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.627502 11105 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 182) took 1753 ms. Trace:
I20260809 01:45:45.627882 11105 rpcz_store.cc:276] 0809 01:45:43.874419 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.504857 (+630438us) service_pool.cc:224] Handling call
0809 01:45:44.783034 (+278177us) spinlock_profiling.cc:243] Waited 3.92 ms on lock 0x7b5800000f98. stack: 00007fe4bc3e2cf3 00007fe4bc3e2bf0 00007fe4bbf90574 0000000000516de4 0000000000516d79 00007fe4bc29d981 00007fe4bc3dd84c 00007fe4bc3dd40b 00007fe4c5321fb9 00007fe4c4563640 00007fe4c4648641 00007fe4c46258d3 00007fe4c45d4210 00007fe4c2b60bf4 00007fe4c2b60b82 00007fe4c2b60ab1
0809 01:45:45.017577 (+234543us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:45.017807 (+ 230us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:45.422102 (+404295us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.626597 (+204495us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:45.627425 (+ 828us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9015680}
I20260809 01:45:45.632360 10589 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_c92dfb9ed1d342f197b158bef20fc0ac" } new_table_name: "default.Table_ae59ae0a1b7d4312856fd310b0c6c8b7" modify_external_catalogs: true
I20260809 01:45:45.643949 10855 tablet_service.cc:1511] Processing CreateTablet for tablet 21e28316ad9442e2bcffba726d13cfe9 (DEFAULT_TABLE table=default.Table_4d3af03d62f44239afda0e4122e17b2d [id=3e8bf3dc52ab433b84eeafb2c2685e21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.645056 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21e28316ad9442e2bcffba726d13cfe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.651444 11553 tablet_bootstrap.cc:654] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:45.658326 10595 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_7b2fc1dac8e941cbb0fd150a175cf8c0" } new_table_name: "default.Table_a86a2cb7b56442e0aed85bbde8eba334" modify_external_catalogs: true
I20260809 01:45:45.662911 10586 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 389407e7ff464e0d8e84bbdbc4c6081e
I20260809 01:45:45.665217 10586 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 249) took 2007 ms. Trace:
I20260809 01:45:45.668758 10591 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 50f45c02a6994c5aaaf9c0a499e15fed
I20260809 01:45:45.670674 10591 catalog_manager.cc:6807] ReplaceTablet: tablet 50f45c02a6994c5aaaf9c0a499e15fed deleted and replaced by tablet ebc209c696ac4c669961f5ed8392af40
I20260809 01:45:45.671181 10591 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:37956 (request call id 260) took 1675 ms. Trace:
I20260809 01:45:45.671545 10591 rpcz_store.cc:276] 0809 01:45:43.995958 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.996213 (+ 255us) service_pool.cc:224] Handling call
0809 01:45:45.671072 (+1674859us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:45.226644 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:45.226954 (+ 310us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:45.226973 (+ 19us) write_op.cc:435] Acquired schema lock
0809 01:45:45.226983 (+ 10us) tablet.cc:662] Decoding operations
0809 01:45:45.227277 (+ 294us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:45.227304 (+ 27us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:45.227322 (+ 18us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:45.227539 (+ 217us) tablet.cc:701] Row locks acquired
0809 01:45:45.227551 (+ 12us) write_op.cc:260] PREPARE: finished
0809 01:45:45.227649 (+ 98us) write_op.cc:270] Start()
0809 01:45:45.227711 (+ 62us) write_op.cc:276] Timestamp: P: 1786239945227537 usec, L: 0
0809 01:45:45.227723 (+ 12us) op_driver.cc:348] REPLICATION: starting
0809 01:45:45.228154 (+ 431us) log.cc:844] Serialized 622 byte log entry
0809 01:45:45.659753 (+431599us) op_driver.cc:464] REPLICATION: finished
0809 01:45:45.665073 (+ 5320us) write_op.cc:301] APPLY: starting
0809 01:45:45.665162 (+ 89us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:45.665385 (+ 223us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:45.666118 (+ 733us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:45.666642 (+ 524us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:45.667028 (+ 386us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:45.667134 (+ 106us) write_op.cc:312] APPLY: finished
0809 01:45:45.667444 (+ 310us) log.cc:844] Serialized 29 byte log entry
0809 01:45:45.667669 (+ 225us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:45.667726 (+ 57us) write_op.cc:454] Released schema lock
0809 01:45:45.667756 (+ 30us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4537,"apply.run_cpu_time_us":2450,"apply.run_wall_time_us":2960,"num_ops":2,"prepare.queue_time_us":2145,"prepare.run_cpu_time_us":1674,"prepare.run_wall_time_us":1674,"replication_time_us":431957}]]}
I20260809 01:45:45.665534 10586 rpcz_store.cc:276] 0809 01:45:43.658260 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.658422 (+ 162us) service_pool.cc:224] Handling call
0809 01:45:43.659943 (+ 1521us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:43.660486 (+ 543us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:43.660601 (+ 115us) catalog_manager.cc:2850] Locking tablets
0809 01:45:45.202947 (+1542346us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:45.662267 (+459320us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:45.662488 (+ 221us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:45.662585 (+ 97us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:45.665147 (+ 2562us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:45.213988 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:45.214315 (+ 327us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:45.214335 (+ 20us) write_op.cc:435] Acquired schema lock
0809 01:45:45.214347 (+ 12us) tablet.cc:662] Decoding operations
0809 01:45:45.214717 (+ 370us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:45.214757 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:45.214777 (+ 20us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:45.215135 (+ 358us) tablet.cc:701] Row locks acquired
0809 01:45:45.215148 (+ 13us) write_op.cc:260] PREPARE: finished
0809 01:45:45.215248 (+ 100us) write_op.cc:270] Start()
0809 01:45:45.215316 (+ 68us) write_op.cc:276] Timestamp: P: 1786239945215133 usec, L: 0
0809 01:45:45.215328 (+ 12us) op_driver.cc:348] REPLICATION: starting
0809 01:45:45.215888 (+ 560us) log.cc:844] Serialized 725 byte log entry
0809 01:45:45.659526 (+443638us) op_driver.cc:464] REPLICATION: finished
0809 01:45:45.659952 (+ 426us) write_op.cc:301] APPLY: starting
0809 01:45:45.660005 (+ 53us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:45.660106 (+ 101us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:45.660183 (+ 77us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:45.660450 (+ 267us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:45.660596 (+ 146us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:45.660634 (+ 38us) write_op.cc:312] APPLY: finished
0809 01:45:45.660902 (+ 268us) log.cc:844] Serialized 29 byte log entry
0809 01:45:45.661920 (+ 1018us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:45.661981 (+ 61us) write_op.cc:454] Released schema lock
0809 01:45:45.662034 (+ 53us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":284,"num_ops":2,"prepare.queue_time_us":3201,"prepare.run_cpu_time_us":2085,"prepare.run_wall_time_us":2085,"replication_time_us":444111}]]}
I20260809 01:45:45.686910 11551 tablet_bootstrap.cc:492] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:45.686882 11105 tablet_service.cc:1558] Processing DeleteTablet for tablet 389407e7ff464e0d8e84bbdbc4c6081e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:43 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:45.687537 11551 ts_tablet_manager.cc:1403] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.785s user 0.018s sys 0.000s
I20260809 01:45:45.688719 11105 tablet_service.cc:1558] Processing DeleteTablet for tablet 50f45c02a6994c5aaaf9c0a499e15fed with delete_type TABLET_DATA_DELETED (replaced by tablet ebc209c696ac4c669961f5ed8392af40) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:45.690019 11118 tablet_service.cc:1558] Processing DeleteTablet for tablet 12fb852b8ea8478cb160a265459865c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 7706838afc9f4dbebc3213b82c4da126) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:45.693504 11551 raft_consensus.cc:359] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:45.694195 11551 raft_consensus.cc:385] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:45.694464 11551 raft_consensus.cc:740] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:45.695065 11551 consensus_queue.cc:260] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:45.695871 11118 tablet_service.cc:1511] Processing CreateTablet for tablet 21e28316ad9442e2bcffba726d13cfe9 (DEFAULT_TABLE table=default.Table_4d3af03d62f44239afda0e4122e17b2d [id=3e8bf3dc52ab433b84eeafb2c2685e21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.697371 11551 ts_tablet_manager.cc:1434] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260809 01:45:45.699254 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21e28316ad9442e2bcffba726d13cfe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.667835 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet 389407e7ff464e0d8e84bbdbc4c6081e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:43 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:45.672607 10864 tablet_service.cc:1558] Processing DeleteTablet for tablet 50f45c02a6994c5aaaf9c0a499e15fed with delete_type TABLET_DATA_DELETED (replaced by tablet ebc209c696ac4c669961f5ed8392af40) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:45.707016 12338 raft_consensus.cc:493] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:45.707665 12338 raft_consensus.cc:515] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:45.712878 11063 leader_election.cc:343] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 201329c676ec4b6abbb72b22519bc41d
I20260809 01:45:45.714148 12338 leader_election.cc:290] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:45.716871 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "201329c676ec4b6abbb72b22519bc41d" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:45.717013 11346 tablet_service.cc:1511] Processing CreateTablet for tablet 6f771c8051784da18a6de9fcf45e0125 (DEFAULT_TABLE table=default.Table_52926e76c3ad4bbcbc425c32d4473a3c [id=51771c6c664942e4b1df1132b5641ffa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.717722 11392 raft_consensus.cc:2468] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:45.718300 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f771c8051784da18a6de9fcf45e0125. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.719623 11062 leader_election.cc:304] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:45.720777 12338 raft_consensus.cc:2804] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:45.721320 12338 raft_consensus.cc:493] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:45.721843 12338 raft_consensus.cc:3060] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:45.722292 11351 tablet_service.cc:1511] Processing CreateTablet for tablet c11b0394533a485294ace9ee5e99bb19 (DEFAULT_TABLE table=default.Table_ab740dbd782f44eb8f423baeede1de0f [id=92630f3e58394023a931a5e2c40e8591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:45.724493 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c11b0394533a485294ace9ee5e99bb19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:45.742784 11551 tablet_bootstrap.cc:492] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:45.751473 11949 ts_tablet_manager.cc:1929] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:45.751838 11949 log.cc:1199] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/91941616dfdd4df3aaa7c8460876d273
I20260809 01:45:45.758741 11354 tablet_service.cc:1558] Processing DeleteTablet for tablet 50f45c02a6994c5aaaf9c0a499e15fed with delete_type TABLET_DATA_DELETED (replaced by tablet ebc209c696ac4c669961f5ed8392af40) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:45.761791 11554 tablet_bootstrap.cc:492] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:45.762452 11554 ts_tablet_manager.cc:1403] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.386s user 0.026s sys 0.024s
I20260809 01:45:45.763510 12338 raft_consensus.cc:515] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:45.767367 11949 ts_tablet_manager.cc:1950] T 91941616dfdd4df3aaa7c8460876d273 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:45.775374 12222 ts_tablet_manager.cc:1929] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260809 01:45:45.775835 11063 leader_election.cc:343] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 201329c676ec4b6abbb72b22519bc41d
I20260809 01:45:45.775998 12222 log.cc:1199] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/c9d839443474498386220c288ff1721b
I20260809 01:45:45.775386 12338 leader_election.cc:290] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:45.777060 12222 ts_tablet_manager.cc:1950] T c9d839443474498386220c288ff1721b P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:45.777758 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "201329c676ec4b6abbb72b22519bc41d" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:45.778702 11392 raft_consensus.cc:3060] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:45.778890 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 189) took 3338 ms. Trace:
I20260809 01:45:45.779240 11949 rpcz_store.cc:276] 0809 01:45:42.440002 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.442041 (+ 2039us) service_pool.cc:224] Handling call
0809 01:45:45.327368 (+2885327us) ts_tablet_manager.cc:1151] Deleting tablet 91941616dfdd4df3aaa7c8460876d273
0809 01:45:45.334651 (+ 7283us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:45.645732 (+311081us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.751363 (+105631us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.778784 (+ 27421us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5476,"tablet-delete.queue_time_us":2826299}
I20260809 01:45:45.781003 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 91941616dfdd4df3aaa7c8460876d273 (table default.Table_ab740dbd782f44eb8f423baeede1de0f [id=92630f3e58394023a931a5e2c40e8591]) successfully deleted
I20260809 01:45:45.781949 11949 tablet_replica.cc:333] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:45.782634 12390 raft_consensus.cc:493] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:45.783733 11949 raft_consensus.cc:2243] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:45.784370 11949 raft_consensus.cc:2272] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:45.783586 12390 raft_consensus.cc:515] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:45.786346 11949 ts_tablet_manager.cc:1916] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:45.788268 12390 leader_election.cc:290] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:45.790413 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0d79e03e66410487a72f1b47b327a0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:45.791882 10804 leader_election.cc:343] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:45.799407 11553 tablet_bootstrap.cc:492] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:45.799947 11553 ts_tablet_manager.cc:1403] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.419s user 0.014s sys 0.002s
I20260809 01:45:45.800478 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 389407e7ff464e0d8e84bbdbc4c6081e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:43 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:45.809417 11553 raft_consensus.cc:359] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:45.810992 12390 raft_consensus.cc:493] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:45.811663 12390 raft_consensus.cc:515] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:45.815414 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac8498dff04b4e93303d7a35e62134" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:45.816030 11551 tablet_bootstrap.cc:654] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:45.817579 10804 leader_election.cc:343] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:45.818820 12390 leader_election.cc:290] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:45.819520 11553 raft_consensus.cc:385] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:45.819831 11553 raft_consensus.cc:740] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:45.821079 10598 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 8323a7a40a1445f980c0adcf6a6d16c7
I20260809 01:45:45.820693 11553 consensus_queue.cc:260] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:45.826889 11553 ts_tablet_manager.cc:1434] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.027s user 0.007s sys 0.003s
I20260809 01:45:45.828078 10598 catalog_manager.cc:6807] ReplaceTablet: tablet 8323a7a40a1445f980c0adcf6a6d16c7 deleted and replaced by tablet 012b043d7eef472eb30bdd8e3eec271f
I20260809 01:45:45.828790 10598 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:37956 (request call id 259) took 1872 ms. Trace:
I20260809 01:45:45.829066 10852 tablet_service.cc:1558] Processing DeleteTablet for tablet 8323a7a40a1445f980c0adcf6a6d16c7 with delete_type TABLET_DATA_DELETED (replaced by tablet 012b043d7eef472eb30bdd8e3eec271f) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:45.830202 11115 tablet_service.cc:1558] Processing DeleteTablet for tablet 8323a7a40a1445f980c0adcf6a6d16c7 with delete_type TABLET_DATA_DELETED (replaced by tablet 012b043d7eef472eb30bdd8e3eec271f) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:45.831171 11553 tablet_bootstrap.cc:492] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:45.829175 10598 rpcz_store.cc:276] 0809 01:45:43.955952 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.956122 (+ 170us) service_pool.cc:224] Handling call
0809 01:45:45.828667 (+1872545us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:45.250050 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:45.250366 (+ 316us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:45.250387 (+ 21us) write_op.cc:435] Acquired schema lock
0809 01:45:45.250397 (+ 10us) tablet.cc:662] Decoding operations
0809 01:45:45.250670 (+ 273us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:45.250699 (+ 29us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:45.250752 (+ 53us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:45.250956 (+ 204us) tablet.cc:701] Row locks acquired
0809 01:45:45.250968 (+ 12us) write_op.cc:260] PREPARE: finished
0809 01:45:45.251058 (+ 90us) write_op.cc:270] Start()
0809 01:45:45.251122 (+ 64us) write_op.cc:276] Timestamp: P: 1786239945250944 usec, L: 0
0809 01:45:45.251135 (+ 13us) op_driver.cc:348] REPLICATION: starting
0809 01:45:45.251605 (+ 470us) log.cc:844] Serialized 622 byte log entry
0809 01:45:45.687672 (+436067us) op_driver.cc:464] REPLICATION: finished
0809 01:45:45.816924 (+129252us) write_op.cc:301] APPLY: starting
0809 01:45:45.817012 (+ 88us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:45.817213 (+ 201us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:45.817451 (+ 238us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:45.818232 (+ 781us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:45.818728 (+ 496us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:45.818821 (+ 93us) write_op.cc:312] APPLY: finished
0809 01:45:45.819129 (+ 308us) log.cc:844] Serialized 29 byte log entry
0809 01:45:45.819376 (+ 247us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:45.819485 (+ 109us) write_op.cc:454] Released schema lock
0809 01:45:45.819535 (+ 50us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:45.822890 (+ 3355us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:45.822956 (+ 66us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":736,"apply.run_cpu_time_us":3018,"apply.run_wall_time_us":4989,"num_ops":2,"prepare.queue_time_us":13700,"prepare.run_cpu_time_us":1720,"prepare.run_wall_time_us":2564,"replication_time_us":436375}]]}
I20260809 01:45:45.836886 10592 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 66cc1253dee24f5d858d521ddbbfb620
I20260809 01:45:45.844830 10852 tablet_service.cc:1558] Processing DeleteTablet for tablet 66cc1253dee24f5d858d521ddbbfb620 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:45 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:45.846021 11553 tablet_bootstrap.cc:654] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:45.800984 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 143) took 4574 ms. Trace:
I20260809 01:45:45.850344 11115 tablet_service.cc:1558] Processing DeleteTablet for tablet 66cc1253dee24f5d858d521ddbbfb620 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:45 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:45.846554 12222 rpcz_store.cc:276] 0809 01:45:41.226468 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.229442 (+ 2974us) service_pool.cc:224] Handling call
0809 01:45:45.551718 (+4322276us) ts_tablet_manager.cc:1151] Deleting tablet c9d839443474498386220c288ff1721b
0809 01:45:45.551770 (+ 52us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:45.670635 (+118865us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.775254 (+104619us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.800879 (+ 25625us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20189,"tablet-delete.queue_time_us":4321778}
I20260809 01:45:45.853410 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet c9d839443474498386220c288ff1721b (table default.Table_0bcdcc37d5f04f40ab4dd02b66baf4a6 [id=97ccdb35be6a437288222feb89f9e751]) successfully deleted
I20260809 01:45:45.855382 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 150) took 4204 ms. Trace:
W20260809 01:45:45.856705 10589 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 consensus state has no leader
I20260809 01:45:45.856043 12222 rpcz_store.cc:276] 0809 01:45:41.650591 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.846592 (+196001us) service_pool.cc:224] Handling call
0809 01:45:45.854590 (+4007998us) ts_tablet_manager.cc:1151] Deleting tablet 60d356fcc48441ca80f2df29b533404f
0809 01:45:45.854730 (+ 140us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:45.855297 (+ 567us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4006871}
W20260809 01:45:45.858337 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 60d356fcc48441ca80f2df29b533404f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60d356fcc48441ca80f2df29b533404f already in progress: creating tablet
I20260809 01:45:45.862392 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 8323a7a40a1445f980c0adcf6a6d16c7 with delete_type TABLET_DATA_DELETED (replaced by tablet 012b043d7eef472eb30bdd8e3eec271f) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:45.864630 11553 tablet_bootstrap.cc:492] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:45.865007 11553 ts_tablet_manager.cc:1403] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.034s user 0.007s sys 0.011s
I20260809 01:45:45.872676 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 151) took 4221 ms. Trace:
I20260809 01:45:45.876014 11551 tablet_bootstrap.cc:492] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:45.876510 11949 ts_tablet_manager.cc:1929] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:45.876542 11551 ts_tablet_manager.cc:1403] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.134s user 0.010s sys 0.010s
I20260809 01:45:45.876840 11949 log.cc:1199] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/cd09bea5263344919ce26593fe958547
I20260809 01:45:45.877430 11949 ts_tablet_manager.cc:1950] T cd09bea5263344919ce26593fe958547 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
W20260809 01:45:45.878813 10581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.125:42063, user_credentials={real_user=slave}} blocked reactor thread for 192356us
I20260809 01:45:45.890152 10589 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_c7666e50daa54efdab20517273b73423" } modify_external_catalogs: true
I20260809 01:45:45.890844 10589 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_c7666e50daa54efdab20517273b73423" } modify_external_catalogs: true
I20260809 01:45:45.897377 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_37259f04b8114af3be8d7a5f8484431d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:45.898885 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37259f04b8114af3be8d7a5f8484431d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:45.906126 11553 raft_consensus.cc:359] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:45.906729 11553 raft_consensus.cc:385] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:45.906945 11553 raft_consensus.cc:740] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:45.909740 11354 tablet_service.cc:1558] Processing DeleteTablet for tablet 66cc1253dee24f5d858d521ddbbfb620 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:45 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:45.913175 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389407e7ff464e0d8e84bbdbc4c6081e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:45.914014 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403cb09c1786491ab0d40502038bf9c0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:45.916167 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0d79e03e66410487a72f1b47b327a0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:45.918970 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac8498dff04b4e93303d7a35e62134" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:45.916891 11553 consensus_queue.cc:260] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:45.928843 11553 ts_tablet_manager.cc:1434] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.064s user 0.004s sys 0.004s
W20260809 01:45:45.951361 10805 leader_election.cc:343] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:45.951946 10805 leader_election.cc:304] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:45.952893 12390 raft_consensus.cc:2749] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:45.955389 10805 leader_election.cc:343] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:45.955773 10805 leader_election.cc:304] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:45.956614 12390 raft_consensus.cc:2749] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:45.956955 10805 leader_election.cc:343] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:45.957429 10805 leader_election.cc:304] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:45.958555 12390 raft_consensus.cc:2749] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:45.966454 10805 leader_election.cc:343] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:45.966454 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7a511512184f53921cea081f82aa37" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:45.866884 11554 raft_consensus.cc:359] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:45.985622 11554 raft_consensus.cc:385] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:45.985942 11554 raft_consensus.cc:740] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:45.986943 11554 consensus_queue.cc:260] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:45.991849 10805 leader_election.cc:304] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:46.003885 12390 raft_consensus.cc:2749] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:45.873133 12222 rpcz_store.cc:276] 0809 01:45:41.650859 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.064796 (+413937us) service_pool.cc:224] Handling call
0809 01:45:45.858072 (+3793276us) ts_tablet_manager.cc:1151] Deleting tablet 995a37a9e6b14889b876da5d76c89b84
0809 01:45:45.858197 (+ 125us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:45.872603 (+ 14406us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3792748}
I20260809 01:45:46.007905 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 153) took 4118 ms. Trace:
I20260809 01:45:46.008363 12222 rpcz_store.cc:276] 0809 01:45:42.047084 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.282156 (+235072us) service_pool.cc:224] Handling call
0809 01:45:46.007089 (+3724933us) ts_tablet_manager.cc:1151] Deleting tablet c528b45de0d14c19af5d6836b7a14f1c
0809 01:45:46.007201 (+ 112us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.007828 (+ 627us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3724233}
W20260809 01:45:46.009862 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 43478.3us
W20260809 01:45:46.011382 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 995a37a9e6b14889b876da5d76c89b84 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 995a37a9e6b14889b876da5d76c89b84 already in progress: creating tablet
I20260809 01:45:46.020267 11551 raft_consensus.cc:359] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:46.023005 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_aa2f13d6f448442aa146a0b68c883765"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:46.021485 11551 raft_consensus.cc:385] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:46.023957 11551 raft_consensus.cc:740] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:46.023113 12222 tablet_replica.cc:333] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
W20260809 01:45:46.024925 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa2f13d6f448442aa146a0b68c883765 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:46.025213 12222 raft_consensus.cc:2243] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:46.025985 12222 raft_consensus.cc:2272] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:46.027000 10805 leader_election.cc:343] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:46.027406 10805 leader_election.cc:304] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:46.028469 12222 ts_tablet_manager.cc:1916] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:46.023409 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet c528b45de0d14c19af5d6836b7a14f1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c528b45de0d14c19af5d6836b7a14f1c already in progress: creating tablet
I20260809 01:45:46.029994 11551 consensus_queue.cc:260] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:46.030468 12390 raft_consensus.cc:2749] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:46.031797 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_5da7b5860d5d457c8fa8e67d7f9833d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:46.037639 11551 ts_tablet_manager.cc:1434] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.161s user 0.006s sys 0.005s
I20260809 01:45:46.038422 11551 tablet_bootstrap.cc:492] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:45.815303 11951 ts_tablet_manager.cc:1929] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:46.043090 11951 log.cc:1199] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/737d7047a0c74a03b3203112987c0cb2
I20260809 01:45:46.044155 11951 ts_tablet_manager.cc:1950] T 737d7047a0c74a03b3203112987c0cb2 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:46.048264 11392 raft_consensus.cc:2468] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:46.051174 10595 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_30d779db722c451baacdc0202550d5b6" } modify_external_catalogs: true
I20260809 01:45:46.050957 11062 leader_election.cc:304] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:46.051720 10595 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_30d779db722c451baacdc0202550d5b6" } modify_external_catalogs: true
I20260809 01:45:46.052667 12338 raft_consensus.cc:2804] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:46.053932 11554 ts_tablet_manager.cc:1434] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.291s user 0.002s sys 0.010s
I20260809 01:45:46.060776 10588 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 85d0a9d73d4841c393e7d2cef41fee89 (table default.Table_a86a2cb7b56442e0aed85bbde8eba334 [id=ac1fb532e0824edaa1a3abc07758fbbe]). Expected version 2 got 1
W20260809 01:45:46.064116 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5da7b5860d5d457c8fa8e67d7f9833d5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:46.067024 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 21e28316ad9442e2bcffba726d13cfe9 (DEFAULT_TABLE table=default.Table_4d3af03d62f44239afda0e4122e17b2d [id=3e8bf3dc52ab433b84eeafb2c2685e21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:46.072427 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21e28316ad9442e2bcffba726d13cfe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:46.077827 11554 tablet_bootstrap.cc:492] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
W20260809 01:45:46.080547 11283 consensus_peers.cc:597] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 12fb852b8ea8478cb160a265459865c4. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:46.086601 11553 tablet_bootstrap.cc:492] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:46.094659 11553 tablet_bootstrap.cc:654] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:46.099273 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 191) took 3658 ms. Trace:
I20260809 01:45:46.113175 11337 tablet_service.cc:1511] Processing CreateTablet for tablet 3cb7f04729a44f71a7750e4d6a975b22 (DEFAULT_TABLE table=default.Table_0ed496dc27464480aceaff109c421877 [id=66e820be55084213ae68a536469f8349]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:46.114393 11337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cb7f04729a44f71a7750e4d6a975b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:46.118333 11346 tablet_service.cc:1511] Processing CreateTablet for tablet a85e82d85424445d94b30402d41c53dc (DEFAULT_TABLE table=default.Table_e09e4f1ce6624e9b91ade11edfcb3e80 [id=979c29485c7d474dbabe82d9ad1ce280]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:46.119450 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a85e82d85424445d94b30402d41c53dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:46.126642 11115 tablet_service.cc:1511] Processing CreateTablet for tablet a85e82d85424445d94b30402d41c53dc (DEFAULT_TABLE table=default.Table_e09e4f1ce6624e9b91ade11edfcb3e80 [id=979c29485c7d474dbabe82d9ad1ce280]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:46.127792 11115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a85e82d85424445d94b30402d41c53dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:46.128378 11554 tablet_bootstrap.cc:654] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:46.114123 10852 tablet_service.cc:1511] Processing CreateTablet for tablet 3cb7f04729a44f71a7750e4d6a975b22 (DEFAULT_TABLE table=default.Table_0ed496dc27464480aceaff109c421877 [id=66e820be55084213ae68a536469f8349]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:46.112540 11553 tablet_bootstrap.cc:492] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:46.139142 11553 ts_tablet_manager.cc:1403] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.053s user 0.010s sys 0.002s
I20260809 01:45:46.141948 11553 raft_consensus.cc:359] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.142632 11553 raft_consensus.cc:385] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:46.143044 11553 raft_consensus.cc:740] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:46.144119 11553 consensus_queue.cc:260] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.099679 11949 rpcz_store.cc:276] 0809 01:45:42.440687 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.470187 (+ 29500us) service_pool.cc:224] Handling call
0809 01:45:45.781815 (+3311628us) ts_tablet_manager.cc:1151] Deleting tablet cd09bea5263344919ce26593fe958547
0809 01:45:45.781851 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:45.801377 (+ 19526us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.876280 (+ 74903us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:46.099215 (+222935us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3239064}
I20260809 01:45:46.151899 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet cd09bea5263344919ce26593fe958547 (table default.Table_6f6560d974c74f2dbfe00a79fc671fcb [id=cb35a16a1e1241abb696b413e3c1d8cd]) successfully deleted
I20260809 01:45:46.165791 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 113) took 7884 ms. Trace:
I20260809 01:45:46.166145 11951 rpcz_store.cc:276] 0809 01:45:38.281559 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.281686 (+ 127us) service_pool.cc:224] Handling call
0809 01:45:45.402337 (+7120651us) ts_tablet_manager.cc:1151] Deleting tablet 737d7047a0c74a03b3203112987c0cb2
0809 01:45:45.402437 (+ 100us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:45.580913 (+178476us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:45.815178 (+234265us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:46.165724 (+350546us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7120012}
I20260809 01:45:46.167344 11553 ts_tablet_manager.cc:1434] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.028s user 0.006s sys 0.001s
I20260809 01:45:46.126762 11118 tablet_service.cc:1511] Processing CreateTablet for tablet 3cb7f04729a44f71a7750e4d6a975b22 (DEFAULT_TABLE table=default.Table_0ed496dc27464480aceaff109c421877 [id=66e820be55084213ae68a536469f8349]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:46.168208 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 737d7047a0c74a03b3203112987c0cb2 (table default.Table_cf7bd2b9aec04c73b8ed72debcab795d [id=ccd0ea07da604f859047f2eca85fc566]) successfully deleted
I20260809 01:45:46.169456 10598 master_service.cc:960] ReplaceTablet: received request to replace tablet f4d58189de244802b299e3b376319c1c from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:46.174070 11951 tablet_replica.cc:333] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:46.173835 10855 tablet_service.cc:1511] Processing CreateTablet for tablet a85e82d85424445d94b30402d41c53dc (DEFAULT_TABLE table=default.Table_e09e4f1ce6624e9b91ade11edfcb3e80 [id=979c29485c7d474dbabe82d9ad1ce280]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:46.177044 11951 raft_consensus.cc:2243] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:46.178720 11951 pending_rounds.cc:70] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316: Trying to abort 1 pending ops.
I20260809 01:45:46.186045 10852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cb7f04729a44f71a7750e4d6a975b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:46.197461 12390 raft_consensus.cc:493] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:46.198303 12390 raft_consensus.cc:515] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.202958 12390 leader_election.cc:290] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:46.204721 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 192) took 3444 ms. Trace:
I20260809 01:45:46.204914 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a85e82d85424445d94b30402d41c53dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:46.205700 11553 tablet_bootstrap.cc:492] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:46.213855 11553 tablet_bootstrap.cc:654] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:46.179524 11951 pending_rounds.cc:77] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438788870103040 op_type: NO_OP noop_request { }
I20260809 01:45:46.222451 11951 raft_consensus.cc:2889] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:46.222936 11951 raft_consensus.cc:2272] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:46.226076 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e4315b3c644846ac572883c51f23de" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:46.228065 10804 leader_election.cc:343] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:46.228325 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e4315b3c644846ac572883c51f23de" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:46.210119 11949 rpcz_store.cc:276] 0809 01:45:42.760267 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.760379 (+ 112us) service_pool.cc:224] Handling call
0809 01:45:46.153298 (+3392919us) ts_tablet_manager.cc:1151] Deleting tablet 36a542c773b7438ead16c5b3af224cf9
0809 01:45:46.204283 (+ 50985us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.204642 (+ 359us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3392294}
W20260809 01:45:46.229635 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 36a542c773b7438ead16c5b3af224cf9 because the tablet was not found. No further retry: Not found: Tablet not found: 36a542c773b7438ead16c5b3af224cf9
I20260809 01:45:46.229892 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 193) took 3450 ms. Trace:
I20260809 01:45:46.230178 11949 rpcz_store.cc:276] 0809 01:45:42.779831 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.782211 (+ 2380us) service_pool.cc:224] Handling call
0809 01:45:46.229665 (+3447454us) ts_tablet_manager.cc:1151] Deleting tablet 0da8d25290814e66b99c2cc1d5f8b0cf
0809 01:45:46.229705 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.229840 (+ 135us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3387960}
W20260809 01:45:46.231302 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 0da8d25290814e66b99c2cc1d5f8b0cf because the tablet was not found. No further retry: Not found: Tablet not found: 0da8d25290814e66b99c2cc1d5f8b0cf
I20260809 01:45:46.231603 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 206) took 3201 ms. Trace:
I20260809 01:45:46.231878 11949 rpcz_store.cc:276] 0809 01:45:43.029973 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.030187 (+ 214us) service_pool.cc:224] Handling call
0809 01:45:46.231310 (+3201123us) ts_tablet_manager.cc:1151] Deleting tablet b3b02bd8c93b43bf90ff0192a9541aae
0809 01:45:46.231347 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.231556 (+ 209us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3200209}
I20260809 01:45:46.233111 11949 tablet_replica.cc:333] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:46.233129 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_aed544389a6747a8aea8e092dbbf896d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:46.234102 11949 raft_consensus.cc:2243] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:46.234608 11949 raft_consensus.cc:2272] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:46.234835 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aed544389a6747a8aea8e092dbbf896d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:46.234963 11951 ts_tablet_manager.cc:1916] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:46.235584 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet b3b02bd8c93b43bf90ff0192a9541aae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b3b02bd8c93b43bf90ff0192a9541aae already in progress: creating tablet
I20260809 01:45:46.236634 11949 ts_tablet_manager.cc:1916] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:46.241803 10805 leader_election.cc:343] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:46.242336 10805 leader_election.cc:304] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:46.243494 12390 raft_consensus.cc:2749] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:46.117945 12338 raft_consensus.cc:697] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:46.248137 12338 consensus_queue.cc:237] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:46.251657 12387 raft_consensus.cc:493] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:46.252552 12387 raft_consensus.cc:515] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:46.259701 12390 raft_consensus.cc:493] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:46.260277 12390 raft_consensus.cc:515] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.264441 12390 leader_election.cc:290] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:46.277896 12390 raft_consensus.cc:493] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:46.278719 12390 raft_consensus.cc:515] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.287935 10589 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 5b4143759f884f3ba34b0f229c3c136d
I20260809 01:45:46.289145 10589 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_50229c26ee0c44adb2b271119f7eabdf" } new_table_name: "default.Table_a1e1833d52934d0b90c8e6f5046d03df" modify_external_catalogs: true
I20260809 01:45:46.293324 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bc7dc3dfc2f4c1286ca24b8e3b20719" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:46.293159 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0354e4c7bf354f22a9c3c389e0e8ec92" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:46.295593 10804 leader_election.cc:343] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:46.296216 10804 leader_election.cc:343] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:46.291885 10597 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_fb9b15123c274d7db877eac3e7906edc [id=266a0465cb4544eba6f3e9df1eacf724] alter complete (version 2)
I20260809 01:45:46.292562 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cb7f04729a44f71a7750e4d6a975b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:46.315310 11951 ts_tablet_manager.cc:1929] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:46.315683 11951 log.cc:1199] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/c795f81d2bc248eeb39b64d6aaa0370c
I20260809 01:45:46.316449 11951 ts_tablet_manager.cc:1950] T c795f81d2bc248eeb39b64d6aaa0370c P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:46.325296 11553 tablet_bootstrap.cc:492] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:46.325814 11553 ts_tablet_manager.cc:1403] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.120s user 0.007s sys 0.006s
I20260809 01:45:46.328256 11553 raft_consensus.cc:359] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.328866 11553 raft_consensus.cc:385] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:46.329118 11553 raft_consensus.cc:740] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:46.332669 12222 ts_tablet_manager.cc:1929] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20260809 01:45:46.334511 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0354e4c7bf354f22a9c3c389e0e8ec92" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:46.336715 10805 leader_election.cc:343] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:46.337149 10805 leader_election.cc:304] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:46.338193 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 118) took 7834 ms. Trace:
I20260809 01:45:46.341605 11951 rpcz_store.cc:276] 0809 01:45:38.503751 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.503909 (+ 158us) service_pool.cc:224] Handling call
0809 01:45:46.173857 (+7669948us) ts_tablet_manager.cc:1151] Deleting tablet c795f81d2bc248eeb39b64d6aaa0370c
0809 01:45:46.173965 (+ 108us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.252112 (+ 78147us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:46.315159 (+ 63047us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:46.338138 (+ 22979us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1910,"tablet-delete.queue_time_us":7609181}
I20260809 01:45:46.345755 12222 log.cc:1199] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/2aac67e7819f4a70b1050b558868f27f
I20260809 01:45:46.343922 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637f0d6261f041b5aeff5fcc709a3dfc" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:46.347163 12222 ts_tablet_manager.cc:1950] T 2aac67e7819f4a70b1050b558868f27f P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:46.346899 11156 raft_consensus.cc:2468] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:46.337119 12300 consensus_queue.cc:1048] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:46.348892 11283 leader_election.cc:304] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:46.349793 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet c795f81d2bc248eeb39b64d6aaa0370c (table default.Table_4eef5dd402bc41168cd7b02c32e84835 [id=ff7d5da48c94470296b8c7154d68388c]) successfully deleted
I20260809 01:45:46.349751 12299 raft_consensus.cc:2804] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:46.344916 12387 leader_election.cc:290] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:46.350293 12299 raft_consensus.cc:493] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:46.350839 12299 raft_consensus.cc:3060] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:46.352813 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 154) took 4284 ms. Trace:
I20260809 01:45:46.353468 12222 rpcz_store.cc:276] 0809 01:45:42.136375 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.282924 (+146549us) service_pool.cc:224] Handling call
0809 01:45:46.022709 (+3739785us) ts_tablet_manager.cc:1151] Deleting tablet 2aac67e7819f4a70b1050b558868f27f
0809 01:45:46.022913 (+ 204us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.299362 (+276449us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:46.332500 (+ 33138us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:46.352724 (+ 20224us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3739089}
I20260809 01:45:46.354482 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 120) took 7698 ms. Trace:
I20260809 01:45:46.356317 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 2aac67e7819f4a70b1050b558868f27f (table default.Table_72e975bfbe594052a24baee87785c71a [id=04d4b772004b44e781c255cf271fab34]) successfully deleted
I20260809 01:45:46.356444 12222 tablet_replica.cc:333] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:46.356107 11951 rpcz_store.cc:276] 0809 01:45:38.655777 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.655934 (+ 157us) service_pool.cc:224] Handling call
0809 01:45:46.354016 (+7698082us) ts_tablet_manager.cc:1151] Deleting tablet 6d35f291c92a45899eed0f2f10a3a1d8
0809 01:45:46.354104 (+ 88us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.354420 (+ 316us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7697519}
I20260809 01:45:46.357970 12222 raft_consensus.cc:2243] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:46.358814 12222 raft_consensus.cc:2272] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:46.358085 11283 consensus_peers.cc:597] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
W20260809 01:45:46.360848 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 6d35f291c92a45899eed0f2f10a3a1d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d35f291c92a45899eed0f2f10a3a1d8 already in progress: creating tablet
I20260809 01:45:46.362390 12222 ts_tablet_manager.cc:1916] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:46.363569 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 124) took 7641 ms. Trace:
I20260809 01:45:46.321411 12418 raft_consensus.cc:493] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:46.376571 12418 raft_consensus.cc:515] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.377507 11951 rpcz_store.cc:276] 0809 01:45:38.722587 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:38.722990 (+ 403us) service_pool.cc:224] Handling call
0809 01:45:46.363158 (+7640168us) ts_tablet_manager.cc:1151] Deleting tablet 4754f2e2a35f4a6a91fa333e2cf564e4
0809 01:45:46.363210 (+ 52us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.363524 (+ 314us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7637628}
W20260809 01:45:46.379632 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 4754f2e2a35f4a6a91fa333e2cf564e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4754f2e2a35f4a6a91fa333e2cf564e4 already in progress: creating tablet
I20260809 01:45:46.382264 10599 master_service.cc:960] ReplaceTablet: received request to replace tablet b84f1943c1a34e999fc7c12a5847cdb6 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:46.383297 11551 tablet_bootstrap.cc:654] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:46.386905 10282 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 116832us
I20260809 01:45:46.390096 11554 tablet_bootstrap.cc:492] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:46.390712 11554 ts_tablet_manager.cc:1403] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.317s user 0.005s sys 0.008s
I20260809 01:45:46.398191 12425 raft_consensus.cc:2749] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:46.398491 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_73716d58e8b348e2a0d3d697b9778fc0" } new_table_name: "default.Table_7229a3cccd684ffeb87a0a510850eab5" modify_external_catalogs: true
I20260809 01:45:46.409929 11951 tablet_replica.cc:333] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:46.410249 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet 637f0d6261f041b5aeff5fcc709a3dfc with delete_type TABLET_DATA_DELETED (replaced by tablet 995a37a9e6b14889b876da5d76c89b84) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:46.410650 11951 raft_consensus.cc:2243] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:46.411159 11951 raft_consensus.cc:2272] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:46.411592 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet c795f81d2bc248eeb39b64d6aaa0370c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:46.412941 11951 ts_tablet_manager.cc:1916] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:46.418503 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7a8b6d06064addb02979a7feedf756" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:46.418675 12390 leader_election.cc:290] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:46.420806 10804 leader_election.cc:343] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:46.422899 10596 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 (table default.Table_ae59ae0a1b7d4312856fd310b0c6c8b7 [id=96595f8bb6b84744bf7fc74f074a51ba]). Expected version 3 got 0
I20260809 01:45:46.428015 11553 consensus_queue.cc:260] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.430075 11553 ts_tablet_manager.cc:1434] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.104s user 0.006s sys 0.000s
I20260809 01:45:46.444413 12299 raft_consensus.cc:515] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:46.447383 12422 tablet.cc:1724] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:46.450731 11554 raft_consensus.cc:359] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.451290 11554 raft_consensus.cc:385] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:46.451524 11554 raft_consensus.cc:740] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:46.452119 11554 consensus_queue.cc:260] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:46.457012 10580 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet c87361b29664464f9fd73be974e2d3a0 (table default.Table_0708cade7ac74a54aedeb8fe16b3b8df [id=74e764815e2242a9be1ca850a0b6ff30], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet c87361b29664464f9fd73be974e2d3a0 consensus state has no leader
I20260809 01:45:46.470589 10595 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 8f0d79e03e66410487a72f1b47b327a0
I20260809 01:45:46.475487 11111 tablet_service.cc:1558] Processing DeleteTablet for tablet 8f0d79e03e66410487a72f1b47b327a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:46.481863 11351 tablet_service.cc:1558] Processing DeleteTablet for tablet 8f0d79e03e66410487a72f1b47b327a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:46.487929 12427 tablet.cc:1724] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:46.492431 10595 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_8031d30fd4be45869a2e535707f3459b" } modify_external_catalogs: true
I20260809 01:45:46.492995 10595 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_8031d30fd4be45869a2e535707f3459b" } modify_external_catalogs: true
I20260809 01:45:46.503784 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637f0d6261f041b5aeff5fcc709a3dfc" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:46.504362 12299 leader_election.cc:290] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:46.504453 11156 raft_consensus.cc:3060] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:46.474470 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet 8f0d79e03e66410487a72f1b47b327a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:46.431137 11553 tablet_bootstrap.cc:492] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
W20260809 01:45:46.437937 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 101247us
I20260809 01:45:46.524364 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bc7dc3dfc2f4c1286ca24b8e3b20719" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:46.525064 11289 leader_election.cc:343] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 637f0d6261f041b5aeff5fcc709a3dfc
W20260809 01:45:46.525925 11289 leader_election.cc:343] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 637f0d6261f041b5aeff5fcc709a3dfc
I20260809 01:45:46.525707 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7a8b6d06064addb02979a7feedf756" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:46.528383 10805 leader_election.cc:343] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:46.528931 10805 leader_election.cc:304] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:46.530227 10805 leader_election.cc:343] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:46.530746 10805 leader_election.cc:304] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:46.533002 12390 raft_consensus.cc:2749] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:46.533041 11156 raft_consensus.cc:2468] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:46.533968 12390 raft_consensus.cc:2749] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:46.475102 12418 leader_election.cc:290] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:46.539094 11351 tablet_service.cc:1558] Processing DeleteTablet for tablet a9d1ed215ba644abbae09e78016b2c21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:46.540527 11554 ts_tablet_manager.cc:1434] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.149s user 0.007s sys 0.000s
I20260809 01:45:46.549420 12338 raft_consensus.cc:493] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:46.549973 12338 raft_consensus.cc:515] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:46.552209 12338 leader_election.cc:290] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:46.553275 11554 tablet_bootstrap.cc:492] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:46.553823 11551 tablet_bootstrap.cc:492] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:46.566987 11551 ts_tablet_manager.cc:1403] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.529s user 0.010s sys 0.007s
I20260809 01:45:46.567060 11283 leader_election.cc:304] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:46.571714 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_5d441fff2d57479ca948ad1d62e2b660"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:46.573293 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d441fff2d57479ca948ad1d62e2b660 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:46.574448 12386 raft_consensus.cc:2804] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:46.575248 12386 raft_consensus.cc:697] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:46.576241 12386 consensus_queue.cc:237] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:46.579746 11553 tablet_bootstrap.cc:654] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:46.581094 11289 consensus_peers.cc:597] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: da86b5e4a27544c59ae6bb387440d246. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:46.586122 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_706bcc86a9534c99b53b27593052f412"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:46.587895 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_706bcc86a9534c99b53b27593052f412 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:46.598454 10598 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet f4d58189de244802b299e3b376319c1c
I20260809 01:45:46.599674 10598 catalog_manager.cc:6807] ReplaceTablet: tablet f4d58189de244802b299e3b376319c1c deleted and replaced by tablet 6dd979cdbf48493581caf81a29947344
W20260809 01:45:46.554200 11062 leader_election.cc:343] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Not found: Tablet not found: ac6eded297214b58a5d8129a60ca1120
W20260809 01:45:46.600955 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 46789us
I20260809 01:45:46.601655 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet f4d58189de244802b299e3b376319c1c with delete_type TABLET_DATA_DELETED (replaced by tablet 6dd979cdbf48493581caf81a29947344) from {username='slave'} at 127.0.0.1:45982
W20260809 01:45:46.602183 11063 leader_election.cc:343] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: ac6eded297214b58a5d8129a60ca1120
I20260809 01:45:46.604153 11063 leader_election.cc:304] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:46.604939 11063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 49504.4us
I20260809 01:45:46.605291 11111 tablet_service.cc:1558] Processing DeleteTablet for tablet f4d58189de244802b299e3b376319c1c with delete_type TABLET_DATA_DELETED (replaced by tablet 6dd979cdbf48493581caf81a29947344) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:46.609004 12338 raft_consensus.cc:2749] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:46.613667 12436 tablet.cc:1724] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:45:46.616519 11351 tablet_service.cc:1558] Processing DeleteTablet for tablet f4d58189de244802b299e3b376319c1c with delete_type TABLET_DATA_DELETED (replaced by tablet 6dd979cdbf48493581caf81a29947344) from {username='slave'} at 127.0.0.1:53476
W20260809 01:45:46.631690 10589 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b9493892c1ff48e7a0558ff0fcb7a0f3 consensus state has no leader
W20260809 01:45:46.644670 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8baa4ae62edf46b5a386661a6826bbab consensus state has no leader
I20260809 01:45:46.646914 11553 tablet_bootstrap.cc:492] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:46.647316 11553 ts_tablet_manager.cc:1403] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.216s user 0.003s sys 0.011s
I20260809 01:45:46.648589 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet ac6eded297214b58a5d8129a60ca1120 with delete_type TABLET_DATA_DELETED (replaced by tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:46.649340 11111 tablet_service.cc:1558] Processing DeleteTablet for tablet 637f0d6261f041b5aeff5fcc709a3dfc with delete_type TABLET_DATA_DELETED (replaced by tablet 995a37a9e6b14889b876da5d76c89b84) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:46.650358 11553 raft_consensus.cc:359] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.650950 11553 raft_consensus.cc:385] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:46.651186 11553 raft_consensus.cc:740] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:46.571290 12222 ts_tablet_manager.cc:1929] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:46.654224 12222 log.cc:1199] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/da86b5e4a27544c59ae6bb387440d246
I20260809 01:45:46.655057 12222 ts_tablet_manager.cc:1950] T da86b5e4a27544c59ae6bb387440d246 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:46.668177 12425 raft_consensus.cc:493] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:46.663370 11553 consensus_queue.cc:260] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.669071 12425 raft_consensus.cc:515] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.669550 12418 raft_consensus.cc:493] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:46.669482 10599 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet b84f1943c1a34e999fc7c12a5847cdb6
I20260809 01:45:46.670145 12418 raft_consensus.cc:515] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.671449 12425 leader_election.cc:290] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:46.671998 10599 catalog_manager.cc:6807] ReplaceTablet: tablet b84f1943c1a34e999fc7c12a5847cdb6 deleted and replaced by tablet 4ed45a2d5e9b4ad8be6f94e4a895003f
I20260809 01:45:46.673216 12418 leader_election.cc:290] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:46.675625 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6c244108a4ff3a2df0585e5280294" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:46.676523 10599 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_1998ec81597a4c4584dc4e4f90caa274" } modify_external_catalogs: true
I20260809 01:45:46.676774 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fffa48ccf58476da79dccd9c008b67c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:46.676931 10599 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_1998ec81597a4c4584dc4e4f90caa274" } modify_external_catalogs: true
W20260809 01:45:46.680092 10805 leader_election.cc:343] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:46.680732 10805 leader_election.cc:343] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:46.686163 11351 tablet_service.cc:1558] Processing DeleteTablet for tablet b84f1943c1a34e999fc7c12a5847cdb6 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ed45a2d5e9b4ad8be6f94e4a895003f) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:46.689152 10855 tablet_service.cc:1558] Processing DeleteTablet for tablet b84f1943c1a34e999fc7c12a5847cdb6 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ed45a2d5e9b4ad8be6f94e4a895003f) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:46.691097 11553 ts_tablet_manager.cc:1434] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.044s user 0.005s sys 0.002s
I20260809 01:45:46.697789 11553 tablet_bootstrap.cc:492] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:46.704038 10595 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 5af35cf31a66474cb0575b2fb1547616
I20260809 01:45:46.706410 11351 tablet_service.cc:1558] Processing DeleteTablet for tablet 5af35cf31a66474cb0575b2fb1547616 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:46.709841 10855 tablet_service.cc:1558] Processing DeleteTablet for tablet 5af35cf31a66474cb0575b2fb1547616 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:46.714922 11554 tablet_bootstrap.cc:654] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:46.720063 11951 ts_tablet_manager.cc:1929] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:46.720412 11951 log.cc:1199] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/b870347b07b04de5b833ab4fd3a2475f
I20260809 01:45:46.721009 11951 ts_tablet_manager.cc:1950] T b870347b07b04de5b833ab4fd3a2475f P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:46.721712 11949 ts_tablet_manager.cc:1929] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:46.722057 11949 log.cc:1199] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/761370ec8b164fd184a2e091030f610b
I20260809 01:45:46.722229 11553 tablet_bootstrap.cc:654] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:46.722718 11949 ts_tablet_manager.cc:1950] T 761370ec8b164fd184a2e091030f610b P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:46.727413 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 128) took 7320 ms. Trace:
I20260809 01:45:46.728387 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fffa48ccf58476da79dccd9c008b67c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:46.730028 10804 leader_election.cc:343] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:46.730465 10804 leader_election.cc:304] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:46.731304 12418 raft_consensus.cc:2749] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:46.731987 11951 rpcz_store.cc:276] 0809 01:45:39.406683 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.406833 (+ 150us) service_pool.cc:224] Handling call
0809 01:45:46.409699 (+7002866us) ts_tablet_manager.cc:1151] Deleting tablet b870347b07b04de5b833ab4fd3a2475f
0809 01:45:46.409785 (+ 86us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.646364 (+236579us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:46.696406 (+ 50042us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:46.727304 (+ 30898us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7002071}
I20260809 01:45:46.734899 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6c244108a4ff3a2df0585e5280294" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:46.736454 11951 tablet_replica.cc:333] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:46.728402 11551 raft_consensus.cc:359] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:46.736938 11551 raft_consensus.cc:385] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:46.737136 11951 raft_consensus.cc:2243] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:46.737212 11551 raft_consensus.cc:740] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:46.738108 11551 consensus_queue.cc:260] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:46.743310 11951 raft_consensus.cc:2272] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:46.745098 11951 ts_tablet_manager.cc:1916] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:46.749892 11115 tablet_service.cc:1558] Processing DeleteTablet for tablet 5af35cf31a66474cb0575b2fb1547616 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:46.657392 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 162) took 3184 ms. Trace:
I20260809 01:45:46.757655 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet b870347b07b04de5b833ab4fd3a2475f (table default.Table_9590b2028c934cbb91c44edf337bb871 [id=a0eb60025e8c435e958f40f1ec5c366c]) successfully deleted
I20260809 01:45:46.758569 11551 ts_tablet_manager.cc:1434] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.158s user 0.005s sys 0.003s
I20260809 01:45:46.759083 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_5c934da8f7cf4e3f91d0964d5984d94d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:46.760768 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5c934da8f7cf4e3f91d0964d5984d94d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:46.757833 12222 rpcz_store.cc:276] 0809 01:45:43.473057 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.727972 (+254915us) service_pool.cc:224] Handling call
0809 01:45:46.356192 (+2628220us) ts_tablet_manager.cc:1151] Deleting tablet da86b5e4a27544c59ae6bb387440d246
0809 01:45:46.356287 (+ 95us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.385283 (+ 28996us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:46.571159 (+185876us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:46.657337 (+ 86178us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2627466}
I20260809 01:45:46.763141 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 159) took 3291 ms. Trace:
I20260809 01:45:46.763486 12222 rpcz_store.cc:276] 0809 01:45:43.472006 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.641137 (+169131us) service_pool.cc:224] Handling call
0809 01:45:46.762774 (+3121637us) ts_tablet_manager.cc:1151] Deleting tablet c16897857df24c6da5744a99d5f687c0
0809 01:45:46.762827 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.763085 (+ 258us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3000758}
I20260809 01:45:46.764729 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 160) took 3292 ms. Trace:
I20260809 01:45:46.764902 11553 tablet_bootstrap.cc:492] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:46.765487 11553 ts_tablet_manager.cc:1403] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.068s user 0.012s sys 0.001s
I20260809 01:45:46.766745 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet da86b5e4a27544c59ae6bb387440d246 (table default.Table_29a4bd6bd10a473dbd9dbda008b9eb16 [id=67ee87fabc7749279341528fbcce16d4]) successfully deleted
W20260809 01:45:46.767445 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet c16897857df24c6da5744a99d5f687c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c16897857df24c6da5744a99d5f687c0 already in progress: creating tablet
I20260809 01:45:46.767992 11551 tablet_bootstrap.cc:492] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:46.770272 11553 raft_consensus.cc:359] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.773695 11553 raft_consensus.cc:385] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:46.774094 11553 raft_consensus.cc:740] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:46.775134 11553 consensus_queue.cc:260] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:46.786468 10804 leader_election.cc:343] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:46.787313 11118 tablet_service.cc:1558] Processing DeleteTablet for tablet b84f1943c1a34e999fc7c12a5847cdb6 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ed45a2d5e9b4ad8be6f94e4a895003f) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:46.786707 11553 ts_tablet_manager.cc:1434] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260809 01:45:46.765041 12222 rpcz_store.cc:276] 0809 01:45:43.472330 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.675812 (+203482us) service_pool.cc:224] Handling call
0809 01:45:46.764375 (+3088563us) ts_tablet_manager.cc:1151] Deleting tablet b2726cb09f734ed18bb9379f94f3f22a
0809 01:45:46.764424 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.764662 (+ 238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19438,"tablet-delete.queue_time_us":3002155}
W20260809 01:45:46.792549 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet b2726cb09f734ed18bb9379f94f3f22a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b2726cb09f734ed18bb9379f94f3f22a already in progress: creating tablet
I20260809 01:45:46.793941 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 167) took 3319 ms. Trace:
I20260809 01:45:46.794417 12222 rpcz_store.cc:276] 0809 01:45:43.474522 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.833711 (+359189us) service_pool.cc:224] Handling call
0809 01:45:46.793411 (+2959700us) ts_tablet_manager.cc:1151] Deleting tablet 91941616dfdd4df3aaa7c8460876d273
0809 01:45:46.793461 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.793866 (+ 405us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2900496}
W20260809 01:45:46.796244 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 91941616dfdd4df3aaa7c8460876d273 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91941616dfdd4df3aaa7c8460876d273 already in progress: creating tablet
I20260809 01:45:46.801215 11553 tablet_bootstrap.cc:492] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:46.809696 10804 leader_election.cc:304] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:46.810401 12418 raft_consensus.cc:2749] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:46.820705 11553 tablet_bootstrap.cc:654] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:46.820529 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_056eba06842f4044b4a9f9922d208c59"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:46.822042 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_056eba06842f4044b4a9f9922d208c59 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:46.824426 10597 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_a86a2cb7b56442e0aed85bbde8eba334 [id=ac1fb532e0824edaa1a3abc07758fbbe] alter complete (version 2)
I20260809 01:45:46.837292 11118 tablet_service.cc:1558] Processing DeleteTablet for tablet 7501d985958746bea58e0ff752d7fed8 with delete_type TABLET_DATA_DELETED (replaced by tablet b2726cb09f734ed18bb9379f94f3f22a) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:46.841148 10595 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_86ddcc80418e437f8fb35426024f6d19" } new_table_name: "default.Table_7222335f5bc84f3992e7574872dfa0de" modify_external_catalogs: true
I20260809 01:45:46.842213 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 207) took 3800 ms. Trace:
I20260809 01:45:46.844810 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 168) took 3370 ms. Trace:
I20260809 01:45:46.845153 12222 rpcz_store.cc:276] 0809 01:45:43.474782 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.893069 (+418287us) service_pool.cc:224] Handling call
0809 01:45:46.844425 (+2951356us) ts_tablet_manager.cc:1151] Deleting tablet cd09bea5263344919ce26593fe958547
0809 01:45:46.844475 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.844750 (+ 275us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2950452}
W20260809 01:45:46.846493 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet cd09bea5263344919ce26593fe958547 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd09bea5263344919ce26593fe958547 already in progress: creating tablet
I20260809 01:45:46.849529 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_0ebfe415d8ef4946b8971a58797d97c6" } new_table_name: "default.Table_f67555499dc34738b62598f70a738d8d" modify_external_catalogs: true
W20260809 01:45:46.872547 10595 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3938bfd21b294fa6a25f0d8efc1ca29a consensus state has no leader
W20260809 01:45:46.874012 11283 consensus_peers.cc:597] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2aac67e7819f4a70b1050b558868f27f. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:46.842607 11949 rpcz_store.cc:276] 0809 01:45:43.042171 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.042404 (+ 233us) service_pool.cc:224] Handling call
0809 01:45:46.232962 (+3190558us) ts_tablet_manager.cc:1151] Deleting tablet 761370ec8b164fd184a2e091030f610b
0809 01:45:46.233000 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.638044 (+405044us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:46.721591 (+ 83547us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:46.842145 (+120554us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3183044}
I20260809 01:45:46.881804 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 761370ec8b164fd184a2e091030f610b (table default.Table_4d3af03d62f44239afda0e4122e17b2d [id=3e8bf3dc52ab433b84eeafb2c2685e21]) successfully deleted
I20260809 01:45:46.888948 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_2b48e5de5f194f2caa53cdd253a8eda4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:46.890156 11554 tablet_bootstrap.cc:492] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
W20260809 01:45:46.890415 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b48e5de5f194f2caa53cdd253a8eda4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:46.890594 11554 ts_tablet_manager.cc:1403] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.338s user 0.014s sys 0.001s
W20260809 01:45:46.892505 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 34616fd5895945b3aa896d5a8ee87249 consensus state has no leader
I20260809 01:45:46.893805 11554 raft_consensus.cc:359] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.894431 11554 raft_consensus.cc:385] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:46.894685 11554 raft_consensus.cc:740] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:46.901628 11553 tablet_bootstrap.cc:492] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:46.902269 11553 ts_tablet_manager.cc:1403] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.101s user 0.013s sys 0.011s
I20260809 01:45:46.906239 11553 raft_consensus.cc:359] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:46.907322 11553 raft_consensus.cc:385] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:46.907768 11553 raft_consensus.cc:740] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:46.909006 11553 consensus_queue.cc:260] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:46.909806 11554 consensus_queue.cc:260] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.914602 11553 ts_tablet_manager.cc:1434] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.012s user 0.009s sys 0.000s
I20260809 01:45:46.916043 11553 tablet_bootstrap.cc:492] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:46.916827 12418 raft_consensus.cc:493] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:46.916836 10595 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_2b01d8bf86d44f96b64752f511b02f50" } modify_external_catalogs: true
I20260809 01:45:46.917375 10595 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_2b01d8bf86d44f96b64752f511b02f50" } modify_external_catalogs: true
I20260809 01:45:46.917333 12418 raft_consensus.cc:515] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:46.920739 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93334cc766044e79d37f293bc8556fb" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:46.921671 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93334cc766044e79d37f293bc8556fb" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:46.922808 10804 leader_election.cc:343] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:46.923480 12418 leader_election.cc:290] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:46.924917 12299 raft_consensus.cc:493] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b5117038a52a45b0aea3d30ca9cfcbe8)
I20260809 01:45:46.925499 12299 raft_consensus.cc:515] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
W20260809 01:45:46.927132 10805 leader_election.cc:343] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:46.927573 10805 leader_election.cc:304] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:46.928311 12425 raft_consensus.cc:2749] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:46.929924 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 208) took 3841 ms. Trace:
I20260809 01:45:46.930266 11949 rpcz_store.cc:276] 0809 01:45:43.088501 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.088669 (+ 168us) service_pool.cc:224] Handling call
0809 01:45:46.929576 (+3840907us) ts_tablet_manager.cc:1151] Deleting tablet bd3662c307ba4ff78b8f3165b23acb1a
0809 01:45:46.929631 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.929867 (+ 236us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3791942}
W20260809 01:45:46.931372 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet bd3662c307ba4ff78b8f3165b23acb1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd3662c307ba4ff78b8f3165b23acb1a already in progress: creating tablet
I20260809 01:45:46.931593 11949 tablet_replica.cc:333] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:46.932349 11949 raft_consensus.cc:2243] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:46.932821 11949 raft_consensus.cc:2272] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:46.934441 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 169) took 3459 ms. Trace:
I20260809 01:45:46.934979 12222 rpcz_store.cc:276] 0809 01:45:43.475051 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.895068 (+420017us) service_pool.cc:224] Handling call
0809 01:45:46.933830 (+3038762us) ts_tablet_manager.cc:1151] Deleting tablet 197b5043f6d348f89853c073b256daa4
0809 01:45:46.933886 (+ 56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.934340 (+ 454us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3037981}
I20260809 01:45:46.939227 12299 leader_election.cc:290] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:46.940258 11951 ts_tablet_manager.cc:1929] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260809 01:45:46.941888 11289 leader_election.cc:343] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 44ac6aecc93447f7935fbd656fd2a86b
W20260809 01:45:46.945971 11283 leader_election.cc:343] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: 44ac6aecc93447f7935fbd656fd2a86b
I20260809 01:45:46.946322 11283 leader_election.cc:304] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:46.947530 11951 log.cc:1199] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/0da8d25290814e66b99c2cc1d5f8b0cf
I20260809 01:45:46.948390 11951 ts_tablet_manager.cc:1950] T 0da8d25290814e66b99c2cc1d5f8b0cf P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:46.951740 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 183) took 3076 ms. Trace:
I20260809 01:45:46.951594 12386 raft_consensus.cc:2749] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:45:46.955044 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 129) took 7548 ms. Trace:
W20260809 01:45:46.957792 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 197b5043f6d348f89853c073b256daa4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 197b5043f6d348f89853c073b256daa4 already in progress: creating tablet
I20260809 01:45:46.950028 11551 tablet_bootstrap.cc:654] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:46.958151 11351 tablet_service.cc:1558] Processing DeleteTablet for tablet 135d3d65ea7846d3bc2cb59d44dbedf6 with delete_type TABLET_DATA_DELETED (replaced by tablet c528b45de0d14c19af5d6836b7a14f1c) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:46.968014 11951 rpcz_store.cc:276] 0809 01:45:39.406995 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.407743 (+ 748us) service_pool.cc:224] Handling call
0809 01:45:46.736304 (+7328561us) ts_tablet_manager.cc:1151] Deleting tablet 0da8d25290814e66b99c2cc1d5f8b0cf
0809 01:45:46.736350 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.806215 (+ 69865us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:46.940144 (+133929us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:46.954985 (+ 14841us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7328003}
I20260809 01:45:46.969480 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 0da8d25290814e66b99c2cc1d5f8b0cf (table default.Table_84e4a5312bbb43c9bd562d0f742537e2 [id=5cc4859efd27438a8570acaafe1e6e78]) successfully deleted
I20260809 01:45:46.978264 11951 tablet_replica.cc:333] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:46.981799 11949 ts_tablet_manager.cc:1916] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:46.952292 12222 rpcz_store.cc:276] 0809 01:45:43.875051 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.712435 (+837384us) service_pool.cc:224] Handling call
0809 01:45:46.951091 (+2238656us) ts_tablet_manager.cc:1151] Deleting tablet b3b02bd8c93b43bf90ff0192a9541aae
0809 01:45:46.951181 (+ 90us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.951650 (+ 469us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2237784}
W20260809 01:45:46.984917 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet b3b02bd8c93b43bf90ff0192a9541aae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b3b02bd8c93b43bf90ff0192a9541aae already in progress: creating tablet
I20260809 01:45:46.979184 11554 ts_tablet_manager.cc:1434] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.088s user 0.005s sys 0.002s
I20260809 01:45:46.991183 11951 raft_consensus.cc:2243] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:46.994009 11951 raft_consensus.cc:2272] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:46.972160 11553 tablet_bootstrap.cc:654] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:47.009595 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 184) took 3113 ms. Trace:
I20260809 01:45:47.009989 11554 tablet_bootstrap.cc:492] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:47.010068 12222 rpcz_store.cc:276] 0809 01:45:43.875683 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.713379 (+837696us) service_pool.cc:224] Handling call
0809 01:45:46.988995 (+2275616us) ts_tablet_manager.cc:1151] Deleting tablet 761370ec8b164fd184a2e091030f610b
0809 01:45:46.989047 (+ 52us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:46.989455 (+ 408us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2263269}
I20260809 01:45:47.019371 11551 tablet_bootstrap.cc:492] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:47.019800 11551 ts_tablet_manager.cc:1403] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.260s user 0.002s sys 0.012s
I20260809 01:45:47.022136 11951 ts_tablet_manager.cc:1916] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:47.023921 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 185) took 3147 ms. Trace:
W20260809 01:45:47.017735 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 761370ec8b164fd184a2e091030f610b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 761370ec8b164fd184a2e091030f610b already in progress: creating tablet
I20260809 01:45:47.032603 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_da48f5f78afb48f09e10d02abccf6cd8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:47.034266 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da48f5f78afb48f09e10d02abccf6cd8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:47.051811 10589 master_service.cc:960] ReplaceTablet: received request to replace tablet 977694496e014ef29f5d7d513e5219c0 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:47.053381 11553 tablet_bootstrap.cc:492] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:47.053800 11553 ts_tablet_manager.cc:1403] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.138s user 0.006s sys 0.004s
I20260809 01:45:47.057710 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_058bb1c98d444d1a8252ad8e07bfbaa4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:47.059325 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_058bb1c98d444d1a8252ad8e07bfbaa4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:47.062350 11554 tablet_bootstrap.cc:654] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:47.063476 12222 rpcz_store.cc:276] 0809 01:45:43.876362 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.725869 (+849507us) service_pool.cc:224] Handling call
0809 01:45:47.022476 (+2296607us) ts_tablet_manager.cc:1151] Deleting tablet bd3662c307ba4ff78b8f3165b23acb1a
0809 01:45:47.022751 (+ 275us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.023772 (+ 1021us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2294940}
I20260809 01:45:47.073076 11553 raft_consensus.cc:359] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.073802 11553 raft_consensus.cc:385] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:47.074178 11553 raft_consensus.cc:740] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:47.074414 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_55fdbc6110a64bed9f6caafd0b681303"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:47.074803 11553 consensus_queue.cc:260] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:47.075798 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55fdbc6110a64bed9f6caafd0b681303 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:47.076207 10591 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_ca43ce2300964cea8a0b69061d48fed0" } new_table_name: "default.Table_3c356f5b797347c1ab98aaca3d06f4eb" modify_external_catalogs: true
I20260809 01:45:47.090641 11553 ts_tablet_manager.cc:1434] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.037s user 0.005s sys 0.000s
I20260809 01:45:47.095597 11553 tablet_bootstrap.cc:492] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:47.096004 10596 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_29e8714d4cdd4d308c5856c9b672239d" } new_table_name: "default.Table_e88b4f26b66d4a609b96ee4747a0deb2" modify_external_catalogs: true
I20260809 01:45:47.113456 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_2ee9bb5a08bb43278f67198743327a71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:47.115429 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ee9bb5a08bb43278f67198743327a71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:47.120450 11949 ts_tablet_manager.cc:1929] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:47.125768 11949 log.cc:1199] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/ef8a334741d3419eaa377338e0eaa889
I20260809 01:45:47.128288 11949 ts_tablet_manager.cc:1950] T ef8a334741d3419eaa377338e0eaa889 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:47.128507 11554 tablet_bootstrap.cc:492] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:47.128899 11554 ts_tablet_manager.cc:1403] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.119s user 0.012s sys 0.007s
I20260809 01:45:47.130797 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 189) took 2899 ms. Trace:
W20260809 01:45:47.130308 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet bd3662c307ba4ff78b8f3165b23acb1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd3662c307ba4ff78b8f3165b23acb1a already in progress: creating tablet
I20260809 01:45:47.131114 12222 rpcz_store.cc:276] 0809 01:45:44.230978 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.906356 (+675378us) service_pool.cc:224] Handling call
0809 01:45:47.130339 (+2223983us) ts_tablet_manager.cc:1151] Deleting tablet ef8a334741d3419eaa377338e0eaa889
0809 01:45:47.130389 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.130733 (+ 344us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2223419}
I20260809 01:45:47.131276 11554 raft_consensus.cc:359] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.131973 11554 raft_consensus.cc:385] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:47.132216 11554 raft_consensus.cc:740] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:47.132680 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 194) took 2873 ms. Trace:
I20260809 01:45:47.132993 12222 rpcz_store.cc:276] 0809 01:45:44.259107 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.914947 (+655840us) service_pool.cc:224] Handling call
0809 01:45:47.132358 (+2217411us) ts_tablet_manager.cc:1151] Deleting tablet 403cb09c1786491ab0d40502038bf9c0
0809 01:45:47.132401 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.132629 (+ 228us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2216921}
I20260809 01:45:47.135587 11951 ts_tablet_manager.cc:1929] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:47.138458 11553 tablet_bootstrap.cc:654] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:47.139714 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 212) took 4005 ms. Trace:
I20260809 01:45:47.141839 12418 raft_consensus.cc:493] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:47.142032 11949 rpcz_store.cc:276] 0809 01:45:43.134275 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.135151 (+ 876us) service_pool.cc:224] Handling call
0809 01:45:46.931448 (+3796297us) ts_tablet_manager.cc:1151] Deleting tablet ef8a334741d3419eaa377338e0eaa889
0809 01:45:46.931489 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.037062 (+105573us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:47.120315 (+ 83253us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:47.139662 (+ 19347us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3795798}
I20260809 01:45:47.142373 12418 raft_consensus.cc:515] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.143344 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet ef8a334741d3419eaa377338e0eaa889 (table default.Table_52926e76c3ad4bbcbc425c32d4473a3c [id=51771c6c664942e4b1df1132b5641ffa]) successfully deleted
I20260809 01:45:47.144848 12418 leader_election.cc:290] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:47.146335 10591 master_service.cc:960] ReplaceTablet: received request to replace tablet ead75d766d0346f6823074d90ba8054e from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:47.151161 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8323a7a40a1445f980c0adcf6a6d16c7" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:47.144161 11949 tablet_replica.cc:333] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
W20260809 01:45:47.152285 10804 leader_election.cc:343] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:47.152793 11949 raft_consensus.cc:2243] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:47.153183 11949 raft_consensus.cc:2272] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:47.153544 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet ef8a334741d3419eaa377338e0eaa889 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef8a334741d3419eaa377338e0eaa889 already in progress: creating tablet
W20260809 01:45:47.154258 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 403cb09c1786491ab0d40502038bf9c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 403cb09c1786491ab0d40502038bf9c0 already in progress: creating tablet
I20260809 01:45:47.154804 11949 ts_tablet_manager.cc:1916] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:47.138132 11554 consensus_queue.cc:260] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.148025 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8323a7a40a1445f980c0adcf6a6d16c7" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:47.162719 10805 leader_election.cc:343] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:47.163075 10805 leader_election.cc:304] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:47.164004 12418 raft_consensus.cc:2749] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:47.165004 11951 log.cc:1199] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/201329c676ec4b6abbb72b22519bc41d
W20260809 01:45:47.165263 10596 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c704d148da8d406aa824253a51b850c8 consensus state has no leader
I20260809 01:45:47.166092 11951 ts_tablet_manager.cc:1950] T 201329c676ec4b6abbb72b22519bc41d P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:47.116748 11551 raft_consensus.cc:359] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.168701 11551 raft_consensus.cc:385] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:47.169039 11551 raft_consensus.cc:740] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:47.169777 11551 consensus_queue.cc:260] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.172014 11551 ts_tablet_manager.cc:1434] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.152s user 0.003s sys 0.005s
I20260809 01:45:47.180078 11554 ts_tablet_manager.cc:1434] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.051s user 0.006s sys 0.001s
I20260809 01:45:47.183751 11553 tablet_bootstrap.cc:492] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:47.183696 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 193) took 2924 ms. Trace:
I20260809 01:45:47.184156 11553 ts_tablet_manager.cc:1403] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.089s user 0.009s sys 0.001s
I20260809 01:45:47.184000 12222 rpcz_store.cc:276] 0809 01:45:44.258869 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.913425 (+654556us) service_pool.cc:224] Handling call
0809 01:45:47.174043 (+2260618us) ts_tablet_manager.cc:1151] Deleting tablet c3bf43978dc94c2b9a7f3c18a4677c40
0809 01:45:47.183345 (+ 9302us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.183634 (+ 289us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2254725}
I20260809 01:45:47.185233 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 130) took 7777 ms. Trace:
I20260809 01:45:47.185390 12222 tablet_replica.cc:333] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:47.186040 12222 raft_consensus.cc:2243] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:47.185922 11951 rpcz_store.cc:276] 0809 01:45:39.407311 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.410213 (+ 2902us) service_pool.cc:224] Handling call
0809 01:45:46.978092 (+7567879us) ts_tablet_manager.cc:1151] Deleting tablet 201329c676ec4b6abbb72b22519bc41d
0809 01:45:46.978155 (+ 63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.105220 (+127065us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:47.135487 (+ 30267us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:47.185107 (+ 49620us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7567331}
I20260809 01:45:47.186693 12222 pending_rounds.cc:70] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7: Trying to abort 1 pending ops.
I20260809 01:45:47.186920 12222 pending_rounds.cc:77] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438820015349760 op_type: NO_OP noop_request { }
I20260809 01:45:47.187255 12222 raft_consensus.cc:2889] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:47.187569 12222 raft_consensus.cc:2272] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:47.187599 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet c3bf43978dc94c2b9a7f3c18a4677c40 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c3bf43978dc94c2b9a7f3c18a4677c40 already in progress: creating tablet
I20260809 01:45:47.188323 11553 raft_consensus.cc:359] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.189379 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 201329c676ec4b6abbb72b22519bc41d (table default.Table_7229a3cccd684ffeb87a0a510850eab5 [id=b15bb9c641894a39a13c4409f4c51545]) successfully deleted
I20260809 01:45:47.196588 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 127) took 7790 ms. Trace:
I20260809 01:45:47.196894 11951 rpcz_store.cc:276] 0809 01:45:39.406260 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.407456 (+ 1196us) service_pool.cc:224] Handling call
0809 01:45:47.186760 (+7779304us) ts_tablet_manager.cc:1151] Deleting tablet 36a542c773b7438ead16c5b3af224cf9
0809 01:45:47.196219 (+ 9459us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.196522 (+ 303us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7772921}
W20260809 01:45:47.198086 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 36a542c773b7438ead16c5b3af224cf9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36a542c773b7438ead16c5b3af224cf9 already in progress: creating tablet
I20260809 01:45:47.200855 10591 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet ead75d766d0346f6823074d90ba8054e
I20260809 01:45:47.199079 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 132) took 7785 ms. Trace:
I20260809 01:45:47.201719 11551 tablet_bootstrap.cc:492] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:47.202348 10591 catalog_manager.cc:6807] ReplaceTablet: tablet ead75d766d0346f6823074d90ba8054e deleted and replaced by tablet e3690212d43543fc982815dcff225a51
I20260809 01:45:47.201845 11951 rpcz_store.cc:276] 0809 01:45:39.413675 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.479567 (+ 65892us) service_pool.cc:224] Handling call
0809 01:45:47.198731 (+7719164us) ts_tablet_manager.cc:1151] Deleting tablet c2f86bba4bd94fbcacdc9c139e9de2d7
0809 01:45:47.198791 (+ 60us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.199034 (+ 243us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7664997}
I20260809 01:45:47.189034 11553 raft_consensus.cc:385] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:47.203393 11553 raft_consensus.cc:740] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:47.203872 10855 tablet_service.cc:1558] Processing DeleteTablet for tablet ead75d766d0346f6823074d90ba8054e with delete_type TABLET_DATA_DELETED (replaced by tablet e3690212d43543fc982815dcff225a51) from {username='slave'} at 127.0.0.1:45982
W20260809 01:45:47.204118 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet c2f86bba4bd94fbcacdc9c139e9de2d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c2f86bba4bd94fbcacdc9c139e9de2d7 already in progress: creating tablet
I20260809 01:45:47.204068 11553 consensus_queue.cc:260] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.205678 11351 tablet_service.cc:1558] Processing DeleteTablet for tablet ead75d766d0346f6823074d90ba8054e with delete_type TABLET_DATA_DELETED (replaced by tablet e3690212d43543fc982815dcff225a51) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:47.208972 11551 tablet_bootstrap.cc:654] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:47.208927 11118 tablet_service.cc:1558] Processing DeleteTablet for tablet ead75d766d0346f6823074d90ba8054e with delete_type TABLET_DATA_DELETED (replaced by tablet e3690212d43543fc982815dcff225a51) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:47.181061 11554 tablet_bootstrap.cc:492] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:47.218070 11553 ts_tablet_manager.cc:1434] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.034s user 0.005s sys 0.003s
I20260809 01:45:47.213366 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 133) took 7653 ms. Trace:
I20260809 01:45:47.218523 11951 rpcz_store.cc:276] 0809 01:45:39.560040 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.560148 (+ 108us) service_pool.cc:224] Handling call
0809 01:45:47.212891 (+7652743us) ts_tablet_manager.cc:1151] Deleting tablet dc230f25baf94cbf80aa8e2771ec23ba
0809 01:45:47.212978 (+ 87us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.213303 (+ 325us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7652262}
W20260809 01:45:47.219830 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet dc230f25baf94cbf80aa8e2771ec23ba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dc230f25baf94cbf80aa8e2771ec23ba already in progress: creating tablet
I20260809 01:45:47.221936 11951 tablet_replica.cc:333] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:47.222652 11951 raft_consensus.cc:2243] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:47.223322 11951 raft_consensus.cc:2272] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:47.230051 12222 ts_tablet_manager.cc:1916] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:47.225459 11951 ts_tablet_manager.cc:1916] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:47.252260 11553 tablet_bootstrap.cc:492] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:47.264000 11949 ts_tablet_manager.cc:1929] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:47.264434 11949 log.cc:1199] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/c3bf43978dc94c2b9a7f3c18a4677c40
I20260809 01:45:47.265107 11949 ts_tablet_manager.cc:1950] T c3bf43978dc94c2b9a7f3c18a4677c40 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:47.245690 11554 tablet_bootstrap.cc:654] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:47.271057 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_25bc6c83b5ad43ef8b4fb22d70970c6a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:47.272744 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25bc6c83b5ad43ef8b4fb22d70970c6a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:47.273972 12468 tablet.cc:1724] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 01:45:47.288522 12436 tablet.cc:1724] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 01:45:47.292470 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_7f67458772ff440387609b94eba60aa4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:47.294152 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f67458772ff440387609b94eba60aa4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:47.298148 11553 tablet_bootstrap.cc:654] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:47.299702 12469 tablet.cc:1724] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 01:45:47.316948 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 216) took 3804 ms. Trace:
I20260809 01:45:47.326499 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_6097f876208540eca5173c69a4e0b6ba" } new_table_name: "default.Table_ee7c1665ed2344278d1ee031c67438a4" modify_external_catalogs: true
I20260809 01:45:47.336254 11551 tablet_bootstrap.cc:492] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:47.336712 11551 ts_tablet_manager.cc:1403] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.164s user 0.005s sys 0.005s
I20260809 01:45:47.339164 11551 raft_consensus.cc:359] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.339905 11551 raft_consensus.cc:385] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:47.340178 11551 raft_consensus.cc:740] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:47.340857 11551 consensus_queue.cc:260] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.341998 12386 raft_consensus.cc:493] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:47.342697 12386 raft_consensus.cc:515] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.344686 12386 leader_election.cc:290] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:47.327229 11553 tablet_bootstrap.cc:492] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
W20260809 01:45:47.353469 11289 leader_election.cc:343] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: a9d1ed215ba644abbae09e78016b2c21
I20260809 01:45:47.353837 11553 ts_tablet_manager.cc:1403] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.102s user 0.010s sys 0.000s
I20260809 01:45:47.356601 11553 raft_consensus.cc:359] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.357575 11553 raft_consensus.cc:385] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:47.357851 11553 raft_consensus.cc:740] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:47.358489 11553 consensus_queue.cc:260] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.364065 11553 ts_tablet_manager.cc:1434] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260809 01:45:47.367632 11553 tablet_bootstrap.cc:492] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:47.372088 11949 rpcz_store.cc:276] 0809 01:45:43.512299 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.512426 (+ 127us) service_pool.cc:224] Handling call
0809 01:45:47.144007 (+3631581us) ts_tablet_manager.cc:1151] Deleting tablet c3bf43978dc94c2b9a7f3c18a4677c40
0809 01:45:47.144046 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.232625 (+ 88579us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:47.263866 (+ 31241us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:47.316834 (+ 52968us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3446801}
I20260809 01:45:47.373499 11949 tablet_replica.cc:333] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:47.373692 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet c3bf43978dc94c2b9a7f3c18a4677c40 (table default.Table_022d458d8ac1457c8e363dfc40bd6880 [id=97f7c7d3e84a4032887e35cf5ea63006]) successfully deleted
I20260809 01:45:47.374607 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_0ed496dc27464480aceaff109c421877" } new_table_name: "default.Table_05de13143d694ea496e94b370bb1ebc8" modify_external_catalogs: true
I20260809 01:45:47.347226 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d1ed215ba644abbae09e78016b2c21" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:47.368427 11118 tablet_service.cc:1511] Processing CreateTablet for tablet fa29cba15cc247debfb8e1907ecd0412 (DEFAULT_TABLE table=default.Table_12598d7dd6e4411793b992a49dbf090a [id=39192535d7454f05b339b6a465b52917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.377782 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa29cba15cc247debfb8e1907ecd0412. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:47.383810 11283 leader_election.cc:343] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:47.384212 11283 leader_election.cc:304] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:47.385951 10855 tablet_service.cc:1511] Processing CreateTablet for tablet fa29cba15cc247debfb8e1907ecd0412 (DEFAULT_TABLE table=default.Table_12598d7dd6e4411793b992a49dbf090a [id=39192535d7454f05b339b6a465b52917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.386970 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa29cba15cc247debfb8e1907ecd0412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.387112 11115 tablet_service.cc:1511] Processing CreateTablet for tablet 2790e2266ade4c698741a940ed49757e (DEFAULT_TABLE table=default.Table_d917f500390043af96190d13685091ca [id=44f4949ef8e74231acfb537d37dbe0ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.388080 11115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2790e2266ade4c698741a940ed49757e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.390259 12387 raft_consensus.cc:2749] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:47.391137 11551 ts_tablet_manager.cc:1434] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.054s user 0.000s sys 0.004s
I20260809 01:45:47.393559 11554 tablet_bootstrap.cc:492] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:47.393949 11554 ts_tablet_manager.cc:1403] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.213s user 0.006s sys 0.009s
I20260809 01:45:47.396314 11554 raft_consensus.cc:359] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.396896 11554 raft_consensus.cc:385] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:47.397131 11554 raft_consensus.cc:740] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:47.383359 12386 raft_consensus.cc:493] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b5117038a52a45b0aea3d30ca9cfcbe8)
I20260809 01:45:47.397959 12386 raft_consensus.cc:515] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } }
I20260809 01:45:47.398984 12222 ts_tablet_manager.cc:1929] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:47.399307 12222 log.cc:1199] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/201329c676ec4b6abbb72b22519bc41d
I20260809 01:45:47.399811 12386 leader_election.cc:290] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:47.399994 12222 ts_tablet_manager.cc:1950] T 201329c676ec4b6abbb72b22519bc41d P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
W20260809 01:45:47.402787 11283 leader_election.cc:343] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: c9d839443474498386220c288ff1721b
I20260809 01:45:47.402819 11351 tablet_service.cc:1511] Processing CreateTablet for tablet fa29cba15cc247debfb8e1907ecd0412 (DEFAULT_TABLE table=default.Table_12598d7dd6e4411793b992a49dbf090a [id=39192535d7454f05b339b6a465b52917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.404349 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa29cba15cc247debfb8e1907ecd0412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.404335 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 197) took 2005 ms. Trace:
I20260809 01:45:47.404606 12222 rpcz_store.cc:276] 0809 01:45:45.398721 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.434462 (+ 35741us) service_pool.cc:224] Handling call
0809 01:45:47.185209 (+1750747us) ts_tablet_manager.cc:1151] Deleting tablet 201329c676ec4b6abbb72b22519bc41d
0809 01:45:47.185277 (+ 68us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.312519 (+127242us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:47.398870 (+ 86351us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:47.404282 (+ 5412us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1750181}
I20260809 01:45:47.406579 11553 tablet_bootstrap.cc:654] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:47.408303 12418 raft_consensus.cc:493] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:47.409042 12418 raft_consensus.cc:515] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:47.410250 11289 leader_election.cc:343] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: c9d839443474498386220c288ff1721b
I20260809 01:45:47.410585 11289 leader_election.cc:304] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:47.411589 12418 leader_election.cc:290] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:47.411629 12335 raft_consensus.cc:2749] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:45:47.410704 11554 consensus_queue.cc:260] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.412819 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 201329c676ec4b6abbb72b22519bc41d (table default.Table_7229a3cccd684ffeb87a0a510850eab5 [id=b15bb9c641894a39a13c4409f4c51545]) successfully deleted
I20260809 01:45:47.415827 11554 ts_tablet_manager.cc:1434] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.022s user 0.004s sys 0.001s
I20260809 01:45:47.419307 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d5a709cd86476594613e788a325094" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:47.419166 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_c79f32e8e08f4d73969d16b8846d500a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:47.420698 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c79f32e8e08f4d73969d16b8846d500a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:47.421234 10805 leader_election.cc:343] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:47.422873 11346 tablet_service.cc:1511] Processing CreateTablet for tablet 2790e2266ade4c698741a940ed49757e (DEFAULT_TABLE table=default.Table_d917f500390043af96190d13685091ca [id=44f4949ef8e74231acfb537d37dbe0ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.422407 10855 tablet_service.cc:1511] Processing CreateTablet for tablet 2790e2266ade4c698741a940ed49757e (DEFAULT_TABLE table=default.Table_d917f500390043af96190d13685091ca [id=44f4949ef8e74231acfb537d37dbe0ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.424081 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2790e2266ade4c698741a940ed49757e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.423929 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d5a709cd86476594613e788a325094" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:47.425294 10804 leader_election.cc:343] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:47.425689 10804 leader_election.cc:304] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:47.426654 12418 raft_consensus.cc:2749] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:47.427692 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2790e2266ade4c698741a940ed49757e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.434105 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_08fb89aaa99c421b9236ced8b7b17362"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:47.435873 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_08fb89aaa99c421b9236ced8b7b17362 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:47.436959 12386 consensus_queue.cc:1048] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 01:45:47.446445 11289 consensus_peers.cc:597] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 637f0d6261f041b5aeff5fcc709a3dfc. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:47.451340 11337 tablet_service.cc:1558] Processing DeleteTablet for tablet 4754f2e2a35f4a6a91fa333e2cf564e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:47.453032 11551 tablet_bootstrap.cc:492] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:47.460741 11949 raft_consensus.cc:2243] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:47.461207 11949 raft_consensus.cc:2272] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:47.463002 11949 ts_tablet_manager.cc:1916] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:47.463676 11111 tablet_service.cc:1511] Processing CreateTablet for tablet c239befb83a6468bb25d5dd3a8f0bcb9 (DEFAULT_TABLE table=default.Table_f10dd364f4ef4a7ab5e1b829e3412ca6 [id=5e03f7e53e1542308b2a1d5e33f9ea0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.465255 11111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c239befb83a6468bb25d5dd3a8f0bcb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.467378 10861 tablet_service.cc:1511] Processing CreateTablet for tablet c239befb83a6468bb25d5dd3a8f0bcb9 (DEFAULT_TABLE table=default.Table_f10dd364f4ef4a7ab5e1b829e3412ca6 [id=5e03f7e53e1542308b2a1d5e33f9ea0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.468786 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c239befb83a6468bb25d5dd3a8f0bcb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.470300 10852 tablet_service.cc:1511] Processing CreateTablet for tablet ebc209c696ac4c669961f5ed8392af40 (DEFAULT_TABLE table=default.Table_eb6df7e248d44109898ab900441bda8b [id=7701d6a8eff047ddac4f8775385c57ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.470325 11337 tablet_service.cc:1511] Processing CreateTablet for tablet c239befb83a6468bb25d5dd3a8f0bcb9 (DEFAULT_TABLE table=default.Table_f10dd364f4ef4a7ab5e1b829e3412ca6 [id=5e03f7e53e1542308b2a1d5e33f9ea0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.471480 10852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebc209c696ac4c669961f5ed8392af40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.472189 11337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c239befb83a6468bb25d5dd3a8f0bcb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.472332 11116 tablet_service.cc:1511] Processing CreateTablet for tablet ebc209c696ac4c669961f5ed8392af40 (DEFAULT_TABLE table=default.Table_eb6df7e248d44109898ab900441bda8b [id=7701d6a8eff047ddac4f8775385c57ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.473842 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebc209c696ac4c669961f5ed8392af40. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:47.482062 10580 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 34616fd5895945b3aa896d5a8ee87249 (table default.Table_f67555499dc34738b62598f70a738d8d [id=946eac74789f4001b253e66f9ff48fe8], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 34616fd5895945b3aa896d5a8ee87249 consensus state has no leader
I20260809 01:45:47.487581 11554 tablet_bootstrap.cc:492] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:47.493304 10588 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_70566a89a3aa4b61884490650a219b50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:47.495077 10588 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_70566a89a3aa4b61884490650a219b50 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:47.499096 11105 tablet_service.cc:1511] Processing CreateTablet for tablet ccf0cebec1d24c6188d48425613c3827 (DEFAULT_TABLE table=default.Table_a36c740bfbe44b4891c5c03b3b684172 [id=07b08aa76d0c45e79f5f78176baa4535]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.501060 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccf0cebec1d24c6188d48425613c3827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.501463 11361 tablet_service.cc:1511] Processing CreateTablet for tablet ebc209c696ac4c669961f5ed8392af40 (DEFAULT_TABLE table=default.Table_eb6df7e248d44109898ab900441bda8b [id=7701d6a8eff047ddac4f8775385c57ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.502595 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebc209c696ac4c669961f5ed8392af40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.508038 11951 ts_tablet_manager.cc:1929] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:45:47.508375 11951 log.cc:1199] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/44ac6aecc93447f7935fbd656fd2a86b
I20260809 01:45:47.509069 11951 ts_tablet_manager.cc:1950] T 44ac6aecc93447f7935fbd656fd2a86b P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:47.531772 12418 raft_consensus.cc:493] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:47.532532 12418 raft_consensus.cc:515] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.536000 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977694496e014ef29f5d7d513e5219c0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:47.538520 11347 tablet_service.cc:1511] Processing CreateTablet for tablet ccf0cebec1d24c6188d48425613c3827 (DEFAULT_TABLE table=default.Table_a36c740bfbe44b4891c5c03b3b684172 [id=07b08aa76d0c45e79f5f78176baa4535]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:47.539002 10804 leader_election.cc:343] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:47.540043 12418 leader_election.cc:290] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:47.541581 10877 tablet_service.cc:1511] Processing CreateTablet for tablet ccf0cebec1d24c6188d48425613c3827 (DEFAULT_TABLE table=default.Table_a36c740bfbe44b4891c5c03b3b684172 [id=07b08aa76d0c45e79f5f78176baa4535]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.542595 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccf0cebec1d24c6188d48425613c3827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.544907 11334 tablet_service.cc:1511] Processing CreateTablet for tablet 4aa1429f21024739b06bec88c3cb24d9 (DEFAULT_TABLE table=default.Table_c36eea2ca6e040ba888f487a8e539e04 [id=1678462d93db45abb2817dcb1278732c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.546345 11110 tablet_service.cc:1511] Processing CreateTablet for tablet 4aa1429f21024739b06bec88c3cb24d9 (DEFAULT_TABLE table=default.Table_c36eea2ca6e040ba888f487a8e539e04 [id=1678462d93db45abb2817dcb1278732c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.547583 11110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4aa1429f21024739b06bec88c3cb24d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.548588 11554 tablet_bootstrap.cc:654] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:47.549453 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 68e6698847804951a7cb2db0ae1bbf94 (DEFAULT_TABLE table=default.Table_218ebdf2aa774da6b7b15e22e0d703df [id=67c1d4eda4df4799b13a15f53bfc1f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.550537 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68e6698847804951a7cb2db0ae1bbf94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.551025 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 68e6698847804951a7cb2db0ae1bbf94 (DEFAULT_TABLE table=default.Table_218ebdf2aa774da6b7b15e22e0d703df [id=67c1d4eda4df4799b13a15f53bfc1f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.575312 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977694496e014ef29f5d7d513e5219c0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:47.576846 10805 leader_election.cc:343] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:47.577232 10805 leader_election.cc:304] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:47.579730 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 00bd9164a6ad4c1780c90fa409acabc1 (DEFAULT_TABLE table=default.Table_20618fa8bbfc4193b40c62c5243d9645 [id=2ba1dfb3e7c54688bf9e85a59fe2a795]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.581008 11347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccf0cebec1d24c6188d48425613c3827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.581219 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 4aa1429f21024739b06bec88c3cb24d9 (DEFAULT_TABLE table=default.Table_c36eea2ca6e040ba888f487a8e539e04 [id=1678462d93db45abb2817dcb1278732c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.582152 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4aa1429f21024739b06bec88c3cb24d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.590269 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_ae40e284667c40089594f8328c9630d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:47.592037 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae40e284667c40089594f8328c9630d7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:47.540503 12418 raft_consensus.cc:493] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:47.595328 12418 raft_consensus.cc:515] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.598340 11334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4aa1429f21024739b06bec88c3cb24d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.600138 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19fe81e277cd4c798a07ceafa0b60566" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:47.601234 10856 tablet_service.cc:1511] Processing CreateTablet for tablet 68e6698847804951a7cb2db0ae1bbf94 (DEFAULT_TABLE table=default.Table_218ebdf2aa774da6b7b15e22e0d703df [id=67c1d4eda4df4799b13a15f53bfc1f19]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:47.601727 10805 leader_election.cc:343] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:47.602581 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68e6698847804951a7cb2db0ae1bbf94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.602710 10856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68e6698847804951a7cb2db0ae1bbf94. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:47.603852 10586 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 75e4315b3c644846ac572883c51f23de consensus state has no leader
I20260809 01:45:47.606073 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19fe81e277cd4c798a07ceafa0b60566" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:47.607597 10804 leader_election.cc:343] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:47.607810 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00bd9164a6ad4c1780c90fa409acabc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.608000 10804 leader_election.cc:304] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:47.609566 12418 leader_election.cc:290] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:47.609839 12335 raft_consensus.cc:493] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:47.610169 12418 raft_consensus.cc:2749] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:47.610460 12335 raft_consensus.cc:515] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.613860 12335 leader_election.cc:290] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:47.614848 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59de95452c0a4945beae5a9344537e0f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:47.616494 11283 leader_election.cc:343] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:47.614867 12418 raft_consensus.cc:2749] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:47.621963 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59de95452c0a4945beae5a9344537e0f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:47.629900 10910 raft_consensus.cc:2468] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
W20260809 01:45:47.630251 10576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.126:42157, user_credentials={real_user=slave}} blocked reactor thread for 70786.1us
I20260809 01:45:47.635501 11289 leader_election.cc:304] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:47.637176 12335 raft_consensus.cc:2804] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:47.637646 12335 raft_consensus.cc:493] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:47.638038 12335 raft_consensus.cc:3060] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:47.646883 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 135) took 7805 ms. Trace:
I20260809 01:45:47.647173 11951 rpcz_store.cc:276] 0809 01:45:39.841195 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:39.841947 (+ 752us) service_pool.cc:224] Handling call
0809 01:45:47.221769 (+7379822us) ts_tablet_manager.cc:1151] Deleting tablet 44ac6aecc93447f7935fbd656fd2a86b
0809 01:45:47.221816 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.306074 (+ 84258us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:47.507883 (+201809us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:47.646827 (+138944us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7379127}
I20260809 01:45:47.648535 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 44ac6aecc93447f7935fbd656fd2a86b (table default.Table_57234dd818414b7397d1ac621bdb06a7 [id=17cbc8bf77e142d08d03bd7a051e00ad]) successfully deleted
I20260809 01:45:47.649387 11951 tablet_replica.cc:333] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:47.650074 11951 raft_consensus.cc:2243] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:47.650595 11951 raft_consensus.cc:2272] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:47.651751 10858 tablet_service.cc:1511] Processing CreateTablet for tablet 00bd9164a6ad4c1780c90fa409acabc1 (DEFAULT_TABLE table=default.Table_20618fa8bbfc4193b40c62c5243d9645 [id=2ba1dfb3e7c54688bf9e85a59fe2a795]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.652338 11951 ts_tablet_manager.cc:1916] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:47.652979 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00bd9164a6ad4c1780c90fa409acabc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.654523 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 205) took 1968 ms. Trace:
I20260809 01:45:47.654875 12222 rpcz_store.cc:276] 0809 01:45:45.686442 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.686582 (+ 140us) service_pool.cc:224] Handling call
0809 01:45:47.654195 (+1967613us) ts_tablet_manager.cc:1151] Deleting tablet 389407e7ff464e0d8e84bbdbc4c6081e
0809 01:45:47.654238 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.654466 (+ 228us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1966552}
W20260809 01:45:47.656247 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 389407e7ff464e0d8e84bbdbc4c6081e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 389407e7ff464e0d8e84bbdbc4c6081e already in progress: creating tablet
I20260809 01:45:47.656603 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 206) took 1968 ms. Trace:
I20260809 01:45:47.656889 12222 rpcz_store.cc:276] 0809 01:45:45.688258 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.688403 (+ 145us) service_pool.cc:224] Handling call
0809 01:45:47.656274 (+1967871us) ts_tablet_manager.cc:1151] Deleting tablet 50f45c02a6994c5aaaf9c0a499e15fed
0809 01:45:47.656322 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.656549 (+ 227us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1966528}
W20260809 01:45:47.660892 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 50f45c02a6994c5aaaf9c0a499e15fed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50f45c02a6994c5aaaf9c0a499e15fed already in progress: creating tablet
I20260809 01:45:47.661361 12386 raft_consensus.cc:493] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:47.661931 12386 raft_consensus.cc:515] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.665901 12386 leader_election.cc:290] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:47.667675 11289 leader_election.cc:343] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 135d3d65ea7846d3bc2cb59d44dbedf6
I20260809 01:45:47.673272 11346 tablet_service.cc:1511] Processing CreateTablet for tablet 117d5fad26fd42dab19cfc1687364506 (DEFAULT_TABLE table=default.Table_a58e460fc9644019b0d51c55b3673fea [id=658a2ccdaf4a4bf180c08191f9eb060d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.674839 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 117d5fad26fd42dab19cfc1687364506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.674710 10864 tablet_service.cc:1511] Processing CreateTablet for tablet 117d5fad26fd42dab19cfc1687364506 (DEFAULT_TABLE table=default.Table_a58e460fc9644019b0d51c55b3673fea [id=658a2ccdaf4a4bf180c08191f9eb060d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.675808 10864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 117d5fad26fd42dab19cfc1687364506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.678174 10873 tablet_service.cc:1511] Processing CreateTablet for tablet 012b043d7eef472eb30bdd8e3eec271f (DEFAULT_TABLE table=default.Table_78c91fd32f104ad785b008e345a24e04 [id=dc279b3b97e3489e803a3408d5d76395]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.679098 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 012b043d7eef472eb30bdd8e3eec271f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.685029 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 012b043d7eef472eb30bdd8e3eec271f (DEFAULT_TABLE table=default.Table_78c91fd32f104ad785b008e345a24e04 [id=dc279b3b97e3489e803a3408d5d76395]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.686112 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 012b043d7eef472eb30bdd8e3eec271f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.687429 11554 tablet_bootstrap.cc:492] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:47.687891 11554 ts_tablet_manager.cc:1403] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.201s user 0.006s sys 0.005s
I20260809 01:45:47.691259 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 977694496e014ef29f5d7d513e5219c0
I20260809 01:45:47.692626 10589 catalog_manager.cc:6807] ReplaceTablet: tablet 977694496e014ef29f5d7d513e5219c0 deleted and replaced by tablet b095338eb59e4161aaaf3afb762cd296
I20260809 01:45:47.693719 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet 977694496e014ef29f5d7d513e5219c0 with delete_type TABLET_DATA_DELETED (replaced by tablet b095338eb59e4161aaaf3afb762cd296) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:47.695149 10865 tablet_service.cc:1558] Processing DeleteTablet for tablet 977694496e014ef29f5d7d513e5219c0 with delete_type TABLET_DATA_DELETED (replaced by tablet b095338eb59e4161aaaf3afb762cd296) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:47.697834 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 325e3d19b83545a197c2e09fc9643b5e (table default.Table_cf7bd2b9aec04c73b8ed72debcab795d [id=ccd0ea07da604f859047f2eca85fc566]). Expected version 2 got 0
I20260809 01:45:47.703368 10599 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 19fe81e277cd4c798a07ceafa0b60566
I20260809 01:45:47.704741 10599 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 282) took 1028 ms. Trace:
I20260809 01:45:47.705211 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet 19fe81e277cd4c798a07ceafa0b60566 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:47.705029 10599 rpcz_store.cc:276] 0809 01:45:46.676180 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.676289 (+ 109us) service_pool.cc:224] Handling call
0809 01:45:46.677301 (+ 1012us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:46.677872 (+ 571us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:46.678008 (+ 136us) catalog_manager.cc:2850] Locking tablets
0809 01:45:47.346094 (+668086us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:47.690656 (+344562us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:47.690840 (+ 184us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:47.703285 (+ 12445us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:47.704683 (+ 1398us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:47.441707 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:47.442075 (+ 368us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:47.442098 (+ 23us) write_op.cc:435] Acquired schema lock
0809 01:45:47.442110 (+ 12us) tablet.cc:662] Decoding operations
0809 01:45:47.442462 (+ 352us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:47.442498 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:47.442521 (+ 23us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:47.442768 (+ 247us) tablet.cc:701] Row locks acquired
0809 01:45:47.442783 (+ 15us) write_op.cc:260] PREPARE: finished
0809 01:45:47.442903 (+ 120us) write_op.cc:270] Start()
0809 01:45:47.443001 (+ 98us) write_op.cc:276] Timestamp: P: 1786239947442845 usec, L: 0
0809 01:45:47.443017 (+ 16us) op_driver.cc:348] REPLICATION: starting
0809 01:45:47.446334 (+ 3317us) log.cc:844] Serialized 725 byte log entry
0809 01:45:47.683317 (+236983us) op_driver.cc:464] REPLICATION: finished
0809 01:45:47.684549 (+ 1232us) write_op.cc:301] APPLY: starting
0809 01:45:47.684623 (+ 74us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:47.684790 (+ 167us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:47.684950 (+ 160us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:47.685307 (+ 357us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:47.685477 (+ 170us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:47.685544 (+ 67us) write_op.cc:312] APPLY: finished
0809 01:45:47.685727 (+ 183us) log.cc:844] Serialized 29 byte log entry
0809 01:45:47.685892 (+ 165us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:47.685955 (+ 63us) write_op.cc:454] Released schema lock
0809 01:45:47.685985 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:47.686251 (+ 266us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:47.686277 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1047,"apply.run_cpu_time_us":1639,"apply.run_wall_time_us":1640,"num_ops":2,"prepare.queue_time_us":93727,"prepare.run_cpu_time_us":2155,"prepare.run_wall_time_us":5896,"raft.queue_time_us":540,"raft.run_cpu_time_us":13,"raft.run_wall_time_us":14,"replication_time_us":240187}]]}
I20260809 01:45:47.710171 10595 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet e93334cc766044e79d37f293bc8556fb
I20260809 01:45:47.716549 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet e93334cc766044e79d37f293bc8556fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:47.719777 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135d3d65ea7846d3bc2cb59d44dbedf6" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:47.720356 11156 raft_consensus.cc:2468] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
W20260809 01:45:47.720463 10591 catalog_manager.cc:5911] Failed to send AlterTable RPC for tablet 325e3d19b83545a197c2e09fc9643b5e (table default.Table_cf7bd2b9aec04c73b8ed72debcab795d [id=ccd0ea07da604f859047f2eca85fc566], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 325e3d19b83545a197c2e09fc9643b5e consensus state has no leader
I20260809 01:45:47.723404 11283 leader_election.cc:304] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:47.724324 12299 raft_consensus.cc:2804] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:47.724740 12299 raft_consensus.cc:493] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:47.725054 12299 raft_consensus.cc:3060] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:47.719224 11554 raft_consensus.cc:359] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.730811 11554 raft_consensus.cc:385] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:47.731273 11554 raft_consensus.cc:740] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:47.732383 12335 raft_consensus.cc:515] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.734864 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 204) took 2049 ms. Trace:
W20260809 01:45:47.735278 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3cb7f04729a44f71a7750e4d6a975b22 consensus state has no leader
I20260809 01:45:47.735176 12222 rpcz_store.cc:276] 0809 01:45:45.685940 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.689790 (+ 3850us) service_pool.cc:224] Handling call
0809 01:45:47.734575 (+2044785us) ts_tablet_manager.cc:1151] Deleting tablet 12fb852b8ea8478cb160a265459865c4
0809 01:45:47.734623 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.734806 (+ 183us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2043936}
I20260809 01:45:47.736589 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 208) took 1908 ms. Trace:
I20260809 01:45:47.736897 12222 rpcz_store.cc:276] 0809 01:45:45.828420 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.829887 (+ 1467us) service_pool.cc:224] Handling call
0809 01:45:47.736292 (+1906405us) ts_tablet_manager.cc:1151] Deleting tablet 8323a7a40a1445f980c0adcf6a6d16c7
0809 01:45:47.736331 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.736540 (+ 209us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1905502}
I20260809 01:45:47.751871 12299 raft_consensus.cc:515] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:47.756788 11289 leader_election.cc:343] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 135d3d65ea7846d3bc2cb59d44dbedf6
W20260809 01:45:47.762192 10576 catalog_manager.cc:5017] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 12fb852b8ea8478cb160a265459865c4 because the tablet was not found. No further retry: Not found: Tablet not found: 12fb852b8ea8478cb160a265459865c4
W20260809 01:45:47.768112 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 8323a7a40a1445f980c0adcf6a6d16c7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8323a7a40a1445f980c0adcf6a6d16c7 already in progress: creating tablet
I20260809 01:45:47.768137 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 209) took 1922 ms. Trace:
I20260809 01:45:47.768567 12222 rpcz_store.cc:276] 0809 01:45:45.846095 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.846362 (+ 267us) service_pool.cc:224] Handling call
0809 01:45:47.767704 (+1921342us) ts_tablet_manager.cc:1151] Deleting tablet 66cc1253dee24f5d858d521ddbbfb620
0809 01:45:47.767758 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.768065 (+ 307us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1916651}
I20260809 01:45:47.770931 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 19fe81e277cd4c798a07ceafa0b60566 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:47.770818 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135d3d65ea7846d3bc2cb59d44dbedf6" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:47.781795 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet 977694496e014ef29f5d7d513e5219c0 with delete_type TABLET_DATA_DELETED (replaced by tablet b095338eb59e4161aaaf3afb762cd296) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:47.783342 11156 raft_consensus.cc:3060] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:47.783470 11125 tablet_service.cc:1558] Processing DeleteTablet for tablet e93334cc766044e79d37f293bc8556fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:47.784107 11113 tablet_service.cc:1558] Processing DeleteTablet for tablet 19fe81e277cd4c798a07ceafa0b60566 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:47.777765 12338 raft_consensus.cc:493] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:47.785960 12338 raft_consensus.cc:515] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.789397 12338 leader_election.cc:290] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:47.791190 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0d79e03e66410487a72f1b47b327a0" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:47.791318 11115 tablet_service.cc:1511] Processing CreateTablet for tablet 012b043d7eef472eb30bdd8e3eec271f (DEFAULT_TABLE table=default.Table_78c91fd32f104ad785b008e345a24e04 [id=dc279b3b97e3489e803a3408d5d76395]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.792865 11115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 012b043d7eef472eb30bdd8e3eec271f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:47.794051 11062 leader_election.cc:343] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:47.772930 12481 raft_consensus.cc:493] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:47.794616 11156 raft_consensus.cc:2468] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:47.795028 12481 raft_consensus.cc:515] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:47.796730 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 66cc1253dee24f5d858d521ddbbfb620 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66cc1253dee24f5d858d521ddbbfb620 already in progress: creating tablet
I20260809 01:45:47.799168 12481 leader_election.cc:290] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:47.801558 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501d985958746bea58e0ff752d7fed8" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:47.801427 11118 tablet_service.cc:1511] Processing CreateTablet for tablet 117d5fad26fd42dab19cfc1687364506 (DEFAULT_TABLE table=default.Table_a58e460fc9644019b0d51c55b3673fea [id=658a2ccdaf4a4bf180c08191f9eb060d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.801793 11554 consensus_queue.cc:260] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.802126 11383 raft_consensus.cc:2468] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:47.803398 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 117d5fad26fd42dab19cfc1687364506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.769821 12222 tablet_replica.cc:333] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:47.804984 12222 raft_consensus.cc:2243] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:47.805408 12222 raft_consensus.cc:2272] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:47.808351 11062 leader_election.cc:304] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:47.813259 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet e93334cc766044e79d37f293bc8556fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:47.813846 11283 leader_election.cc:304] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:47.814723 12387 raft_consensus.cc:2804] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:47.814872 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0d79e03e66410487a72f1b47b327a0" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:47.815258 12387 raft_consensus.cc:697] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:47.815536 10905 raft_consensus.cc:2468] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:47.816095 12387 consensus_queue.cc:237] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:47.818186 11063 leader_election.cc:343] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 7501d985958746bea58e0ff752d7fed8
I20260809 01:45:47.819033 11063 leader_election.cc:304] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:47.820941 11553 tablet_bootstrap.cc:492] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:47.821413 11553 ts_tablet_manager.cc:1403] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.454s user 0.010s sys 0.007s
I20260809 01:45:47.824174 11553 raft_consensus.cc:359] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.824843 11553 raft_consensus.cc:385] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:47.825167 11553 raft_consensus.cc:740] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:47.826622 11553 consensus_queue.cc:260] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.826853 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_058bb1c98d444d1a8252ad8e07bfbaa4" } new_table_name: "default.Table_4f9017aa8338441e84bc6913c4915e42" modify_external_catalogs: true
I20260809 01:45:47.830654 11554 ts_tablet_manager.cc:1434] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.142s user 0.007s sys 0.001s
I20260809 01:45:47.831651 11554 tablet_bootstrap.cc:492] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:47.835500 12222 ts_tablet_manager.cc:1916] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:47.819536 11063 raft_consensus.cc:2686] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 01:45:47.757273 12299 leader_election.cc:290] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:47.838636 11551 tablet_bootstrap.cc:654] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:47.848502 12299 raft_consensus.cc:493] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:47.848968 12299 raft_consensus.cc:515] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.852864 12299 leader_election.cc:290] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:47.854120 11289 leader_election.cc:343] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 4754f2e2a35f4a6a91fa333e2cf564e4
I20260809 01:45:47.855414 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59de95452c0a4945beae5a9344537e0f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:47.855540 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4754f2e2a35f4a6a91fa333e2cf564e4" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:47.856829 11283 leader_election.cc:343] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:47.857726 11283 leader_election.cc:343] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:47.858055 11283 leader_election.cc:304] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:47.838231 12481 raft_consensus.cc:2804] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:47.859211 12481 raft_consensus.cc:493] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:47.859735 12481 raft_consensus.cc:3060] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:47.850490 12482 raft_consensus.cc:493] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:47.861660 12299 raft_consensus.cc:2749] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:47.862022 12482 raft_consensus.cc:515] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:47.864147 12482 leader_election.cc:290] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:47.851974 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59de95452c0a4945beae5a9344537e0f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:47.864959 10905 raft_consensus.cc:3060] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:47.866372 10586 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f1d5fdaaed9b40ceab053797439a38c9 has no consensus state
I20260809 01:45:47.871243 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325e3d19b83545a197c2e09fc9643b5e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:47.880640 10805 leader_election.cc:343] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:47.888062 10905 raft_consensus.cc:2468] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:47.888716 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325e3d19b83545a197c2e09fc9643b5e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:47.890304 11110 tablet_service.cc:1558] Processing DeleteTablet for tablet 135d3d65ea7846d3bc2cb59d44dbedf6 with delete_type TABLET_DATA_DELETED (replaced by tablet c528b45de0d14c19af5d6836b7a14f1c) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:47.852429 12335 leader_election.cc:290] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:47.881498 11554 tablet_bootstrap.cc:654] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:47.895736 10804 leader_election.cc:343] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:47.907455 10804 leader_election.cc:304] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:47.908552 12482 raft_consensus.cc:2749] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:47.909613 11289 leader_election.cc:304] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:47.911180 12299 raft_consensus.cc:2804] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:47.911549 11951 ts_tablet_manager.cc:1929] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:45:47.911972 11951 log.cc:1199] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/c9d839443474498386220c288ff1721b
I20260809 01:45:47.912786 12299 raft_consensus.cc:697] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:47.913671 11951 ts_tablet_manager.cc:1950] T c9d839443474498386220c288ff1721b P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:47.911229 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 00d27eecce7447eda6771b7bd0196193 (DEFAULT_TABLE table=default.Table_b29e016ecc9343689732c208bfd2126e [id=cba3f1458c6a44ecaabbf92be35133f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.913806 12299 consensus_queue.cc:237] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:47.917755 10586 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_ff0d273341b54d79b4e634ea694e982f" } modify_external_catalogs: true
I20260809 01:45:47.919268 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00d27eecce7447eda6771b7bd0196193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.926458 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 00bd9164a6ad4c1780c90fa409acabc1 (DEFAULT_TABLE table=default.Table_20618fa8bbfc4193b40c62c5243d9645 [id=2ba1dfb3e7c54688bf9e85a59fe2a795]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.927518 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00bd9164a6ad4c1780c90fa409acabc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.904121 11346 tablet_service.cc:1511] Processing CreateTablet for tablet 00d27eecce7447eda6771b7bd0196193 (DEFAULT_TABLE table=default.Table_b29e016ecc9343689732c208bfd2126e [id=cba3f1458c6a44ecaabbf92be35133f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.929849 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00d27eecce7447eda6771b7bd0196193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.940825 11110 tablet_service.cc:1511] Processing CreateTablet for tablet 8a02b2e4506b41d995d66a6e35f9b050 (DEFAULT_TABLE table=default.Table_8edd49df5953499184f2a39b02c69e91 [id=ad69ee9c03f34dcfb36237ba84945e83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:47.942598 11110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a02b2e4506b41d995d66a6e35f9b050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:47.917850 10598 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_5c33dd5878b34a619bf65e590d9aca96" } modify_external_catalogs: true
I20260809 01:45:47.946496 10598 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_5c33dd5878b34a619bf65e590d9aca96" } modify_external_catalogs: true
I20260809 01:45:47.946298 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_01317dd35d864bfbae02d0d3a30e31cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:47.946903 10588 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 8baa4ae62edf46b5a386661a6826bbab (table default.Table_7229a3cccd684ffeb87a0a510850eab5 [id=b15bb9c641894a39a13c4409f4c51545]). Expected version 3 got 0
W20260809 01:45:47.947750 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01317dd35d864bfbae02d0d3a30e31cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:47.948413 10586 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_ff0d273341b54d79b4e634ea694e982f" } modify_external_catalogs: true
I20260809 01:45:47.945295 10596 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_36b7ad5cff5e46d99d519da60b890733" } new_table_name: "default.Table_0908ea7070134c749de4ff154f23628e" modify_external_catalogs: true
I20260809 01:45:47.950240 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_11fa48659a604ed6826b8f83c7580660"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:47.955044 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11fa48659a604ed6826b8f83c7580660 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:47.945070 11553 ts_tablet_manager.cc:1434] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.123s user 0.004s sys 0.001s
I20260809 01:45:47.861575 11949 ts_tablet_manager.cc:1929] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:47.968003 11949 log.cc:1199] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/403cb09c1786491ab0d40502038bf9c0
I20260809 01:45:47.968705 11949 ts_tablet_manager.cc:1950] T 403cb09c1786491ab0d40502038bf9c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:47.980532 10598 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 750aa437ef594587a512a9c9c8278aa8
I20260809 01:45:48.006459 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 217) took 4393 ms. Trace:
I20260809 01:45:48.009167 11347 tablet_service.cc:1511] Processing CreateTablet for tablet 8a02b2e4506b41d995d66a6e35f9b050 (DEFAULT_TABLE table=default.Table_8edd49df5953499184f2a39b02c69e91 [id=ad69ee9c03f34dcfb36237ba84945e83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:48.011096 11347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a02b2e4506b41d995d66a6e35f9b050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:48.015841 11553 tablet_bootstrap.cc:492] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:48.015434 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_e703629aad544e88a395f30f87f6f139"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:48.016067 10596 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b095338eb59e4161aaaf3afb762cd296 has no consensus state
W20260809 01:45:48.016947 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e703629aad544e88a395f30f87f6f139 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:48.021155 10596 catalog_manager.cc:5719] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67). New cstate: current_term: 1 leader_uuid: "4f60c9ac9414403d9ea5a97b103c2316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:48.024024 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 156) took 6782 ms. Trace:
I20260809 01:45:48.024369 11951 rpcz_store.cc:276] 0809 01:45:41.241416 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.335314 (+ 93898us) service_pool.cc:224] Handling call
0809 01:45:47.649243 (+6313929us) ts_tablet_manager.cc:1151] Deleting tablet c9d839443474498386220c288ff1721b
0809 01:45:47.649285 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.794322 (+145037us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:47.911122 (+116800us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.023966 (+112844us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6285684}
I20260809 01:45:48.025732 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet c9d839443474498386220c288ff1721b (table default.Table_0bcdcc37d5f04f40ab4dd02b66baf4a6 [id=97ccdb35be6a437288222feb89f9e751]) successfully deleted
I20260809 01:45:48.030665 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_946a0fc83ebc47b99a4d35abd513a966"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:48.032279 11554 tablet_bootstrap.cc:492] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
W20260809 01:45:48.032513 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_946a0fc83ebc47b99a4d35abd513a966 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:48.038257 11554 ts_tablet_manager.cc:1403] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.207s user 0.005s sys 0.008s
I20260809 01:45:48.006798 11949 rpcz_store.cc:276] 0809 01:45:43.613510 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.613636 (+ 126us) service_pool.cc:224] Handling call
0809 01:45:47.373370 (+3759734us) ts_tablet_manager.cc:1151] Deleting tablet 403cb09c1786491ab0d40502038bf9c0
0809 01:45:47.373406 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.714843 (+341437us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:47.859740 (+144897us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.006381 (+146641us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122072,"tablet-delete.queue_time_us":3675517}
I20260809 01:45:48.039721 11949 tablet_replica.cc:333] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:48.040467 11949 raft_consensus.cc:2243] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:48.034226 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 163) took 6158 ms. Trace:
I20260809 01:45:48.042434 11951 rpcz_store.cc:276] 0809 01:45:41.876161 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.024841 (+148680us) service_pool.cc:224] Handling call
0809 01:45:48.033775 (+6008934us) ts_tablet_manager.cc:1151] Deleting tablet 995a37a9e6b14889b876da5d76c89b84
0809 01:45:48.033828 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.034172 (+ 344us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5976868}
W20260809 01:45:48.044268 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 995a37a9e6b14889b876da5d76c89b84 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 995a37a9e6b14889b876da5d76c89b84 already in progress: creating tablet
I20260809 01:45:48.047672 12222 ts_tablet_manager.cc:1929] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:48.048800 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 162) took 6174 ms. Trace:
I20260809 01:45:48.058305 12222 log.cc:1199] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/8f0d79e03e66410487a72f1b47b327a0
I20260809 01:45:48.061697 11951 rpcz_store.cc:276] 0809 01:45:41.874411 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:41.874667 (+ 256us) service_pool.cc:224] Handling call
0809 01:45:48.048383 (+6173716us) ts_tablet_manager.cc:1151] Deleting tablet 60d356fcc48441ca80f2df29b533404f
0809 01:45:48.048438 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.048736 (+ 298us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5910780}
I20260809 01:45:48.067189 10586 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 72b4850de7d740d794283ca93152fdc1
I20260809 01:45:48.072489 10592 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_7229a3cccd684ffeb87a0a510850eab5" } new_table_name: "default.Table_fba7d62f160e436c9d150c884a5e013a" modify_external_catalogs: true
W20260809 01:45:48.072747 12482 raft_consensus.cc:670] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopping
I20260809 01:45:48.073658 12222 ts_tablet_manager.cc:1950] T 8f0d79e03e66410487a72f1b47b327a0 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:48.069829 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 750aa437ef594587a512a9c9c8278aa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:47 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:48.072623 11337 tablet_service.cc:1558] Processing DeleteTablet for tablet 72b4850de7d740d794283ca93152fdc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:47 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:48.075749 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 403cb09c1786491ab0d40502038bf9c0 (table default.Table_63e5e407241d4eaea93f27d8285ab33a [id=9e066a2a246442b78bfc003520f4cfbd]) successfully deleted
I20260809 01:45:48.077080 11949 raft_consensus.cc:2272] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:48.076988 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 60d356fcc48441ca80f2df29b533404f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60d356fcc48441ca80f2df29b533404f already in progress: creating tablet
I20260809 01:45:48.078730 11949 ts_tablet_manager.cc:1916] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:48.082182 11951 tablet_replica.cc:333] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:48.082933 11951 raft_consensus.cc:2243] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:48.083813 11951 raft_consensus.cc:2272] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:48.084765 10586 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 8baa4ae62edf46b5a386661a6826bbab (table default.Table_7229a3cccd684ffeb87a0a510850eab5 [id=b15bb9c641894a39a13c4409f4c51545]). Expected version 3 got 0
I20260809 01:45:48.089978 10597 master_service.cc:960] ReplaceTablet: received request to replace tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:48.102167 10855 tablet_service.cc:1558] Processing DeleteTablet for tablet 72b4850de7d740d794283ca93152fdc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:47 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:48.103348 10877 tablet_service.cc:1511] Processing CreateTablet for tablet 8a02b2e4506b41d995d66a6e35f9b050 (DEFAULT_TABLE table=default.Table_8edd49df5953499184f2a39b02c69e91 [id=ad69ee9c03f34dcfb36237ba84945e83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:48.104545 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a02b2e4506b41d995d66a6e35f9b050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:48.061169 11554 raft_consensus.cc:359] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.106633 11554 raft_consensus.cc:385] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:48.106978 11554 raft_consensus.cc:740] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:48.113051 10852 tablet_service.cc:1558] Processing DeleteTablet for tablet 750aa437ef594587a512a9c9c8278aa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:47 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:48.116748 11951 ts_tablet_manager.cc:1916] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:48.115140 11554 consensus_queue.cc:260] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.137986 11553 tablet_bootstrap.cc:654] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:48.147830 11554 ts_tablet_manager.cc:1434] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.107s user 0.001s sys 0.007s
I20260809 01:45:48.150417 11105 tablet_service.cc:1511] Processing CreateTablet for tablet 00d27eecce7447eda6771b7bd0196193 (DEFAULT_TABLE table=default.Table_b29e016ecc9343689732c208bfd2126e [id=cba3f1458c6a44ecaabbf92be35133f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:48.142516 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 212) took 1667 ms. Trace:
I20260809 01:45:48.165845 12222 rpcz_store.cc:276] 0809 01:45:46.474931 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.475112 (+ 181us) service_pool.cc:224] Handling call
0809 01:45:47.769649 (+1294537us) ts_tablet_manager.cc:1151] Deleting tablet 8f0d79e03e66410487a72f1b47b327a0
0809 01:45:47.769700 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:47.935121 (+165421us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.047499 (+112378us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.142337 (+ 94838us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1293672}
I20260809 01:45:48.173758 11554 tablet_bootstrap.cc:492] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:48.174126 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet 750aa437ef594587a512a9c9c8278aa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:47 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:48.179226 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 213) took 1574 ms. Trace:
I20260809 01:45:48.179550 12222 rpcz_store.cc:276] 0809 01:45:46.604724 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.604943 (+ 219us) service_pool.cc:224] Handling call
0809 01:45:48.166775 (+1561832us) ts_tablet_manager.cc:1151] Deleting tablet f4d58189de244802b299e3b376319c1c
0809 01:45:48.178935 (+ 12160us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.179173 (+ 238us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1560878}
I20260809 01:45:48.183034 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 8f0d79e03e66410487a72f1b47b327a0 (table default.Table_30d779db722c451baacdc0202550d5b6 [id=869fc1e71c7142c5bbd1fa358c94e453]) successfully deleted
W20260809 01:45:48.183681 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet f4d58189de244802b299e3b376319c1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f4d58189de244802b299e3b376319c1c already in progress: creating tablet
I20260809 01:45:48.184414 11118 tablet_service.cc:1511] Processing CreateTablet for tablet d146bdaf4dd2418ca870c4afeb6ccb6c (DEFAULT_TABLE table=default.Table_62a39e76a73049dea65f8948c5513aa1 [id=4070cd0e58944571831808565b658bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:48.187577 10588 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_5264c640dfc04d9bae010805cdaf4ea6" } modify_external_catalogs: true
I20260809 01:45:48.188050 10588 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_5264c640dfc04d9bae010805cdaf4ea6" } modify_external_catalogs: true
I20260809 01:45:48.188630 11553 tablet_bootstrap.cc:492] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:48.189198 11553 ts_tablet_manager.cc:1403] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.175s user 0.011s sys 0.007s
I20260809 01:45:48.190518 11551 tablet_bootstrap.cc:492] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:48.190881 11551 ts_tablet_manager.cc:1403] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.738s user 0.007s sys 0.006s
I20260809 01:45:48.192499 11553 raft_consensus.cc:359] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.193081 11553 raft_consensus.cc:385] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:48.193315 11553 raft_consensus.cc:740] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:48.193876 11553 consensus_queue.cc:260] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.195222 11551 raft_consensus.cc:359] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.195892 11551 raft_consensus.cc:385] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:48.196205 11551 raft_consensus.cc:740] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:48.196884 11551 consensus_queue.cc:260] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.199999 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_1ac0a8ad913b419188050971d1b6bf6b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:48.202420 11553 ts_tablet_manager.cc:1434] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260809 01:45:48.181708 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00d27eecce7447eda6771b7bd0196193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:48.203461 11553 tablet_bootstrap.cc:492] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:48.180789 12222 tablet_replica.cc:333] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:48.205749 11366 tablet_service.cc:1511] Processing CreateTablet for tablet d146bdaf4dd2418ca870c4afeb6ccb6c (DEFAULT_TABLE table=default.Table_62a39e76a73049dea65f8948c5513aa1 [id=4070cd0e58944571831808565b658bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:48.206676 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d146bdaf4dd2418ca870c4afeb6ccb6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:48.206549 12222 raft_consensus.cc:2243] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:48.207182 12222 raft_consensus.cc:2272] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:48.214829 11553 tablet_bootstrap.cc:654] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:48.231024 11553 tablet_bootstrap.cc:492] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:48.231467 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d146bdaf4dd2418ca870c4afeb6ccb6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:48.231559 11553 ts_tablet_manager.cc:1403] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.028s user 0.016s sys 0.006s
I20260809 01:45:48.209458 12222 ts_tablet_manager.cc:1916] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:48.232707 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet 72b4850de7d740d794283ca93152fdc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:47 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:48.246933 11554 tablet_bootstrap.cc:654] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:48.230206 12481 raft_consensus.cc:515] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:48.270974 12481 leader_election.cc:290] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:48.274175 12222 ts_tablet_manager.cc:1929] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:48.274669 12222 log.cc:1199] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/ac6eded297214b58a5d8129a60ca1120
I20260809 01:45:48.275575 12222 ts_tablet_manager.cc:1950] T ac6eded297214b58a5d8129a60ca1120 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:48.278043 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 215) took 1629 ms. Trace:
I20260809 01:45:48.278432 12222 rpcz_store.cc:276] 0809 01:45:46.648230 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.648453 (+ 223us) service_pool.cc:224] Handling call
0809 01:45:48.180578 (+1532125us) ts_tablet_manager.cc:1151] Deleting tablet ac6eded297214b58a5d8129a60ca1120
0809 01:45:48.180637 (+ 59us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.256319 (+ 75682us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.274005 (+ 17686us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.277959 (+ 3954us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1531538}
I20260809 01:45:48.279819 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet ac6eded297214b58a5d8129a60ca1120 (table default.Table_ae59ae0a1b7d4312856fd310b0c6c8b7 [id=96595f8bb6b84744bf7fc74f074a51ba]) successfully deleted
I20260809 01:45:48.281615 11551 ts_tablet_manager.cc:1434] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.090s user 0.004s sys 0.006s
W20260809 01:45:48.291430 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ac0a8ad913b419188050971d1b6bf6b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:48.320303 10599 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_e88b4f26b66d4a609b96ee4747a0deb2" } new_table_name: "default.Table_aed078d7fcc9482aa4c52c4664e16dc2" modify_external_catalogs: true
I20260809 01:45:48.298559 12222 tablet_replica.cc:333] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:48.375630 11551 tablet_bootstrap.cc:492] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:48.387681 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_836b8d3fc2e34cb786cadfc1794cb1ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:48.387879 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_99228426aa9e4d1b84a5f23af6de8669"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:48.389221 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_836b8d3fc2e34cb786cadfc1794cb1ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:48.389233 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_99228426aa9e4d1b84a5f23af6de8669 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:48.396006 10596 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_62a39e76a73049dea65f8948c5513aa1" } modify_external_catalogs: true
I20260809 01:45:48.396500 10596 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_62a39e76a73049dea65f8948c5513aa1" } modify_external_catalogs: true
W20260809 01:45:48.402567 10592 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8baa4ae62edf46b5a386661a6826bbab consensus state has no leader
I20260809 01:45:48.424392 11951 ts_tablet_manager.cc:1929] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:48.424754 11951 log.cc:1199] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/12fb852b8ea8478cb160a265459865c4
I20260809 01:45:48.270874 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501d985958746bea58e0ff752d7fed8" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:48.426014 11383 raft_consensus.cc:3060] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:48.408215 11110 tablet_service.cc:1558] Processing DeleteTablet for tablet 389407e7ff464e0d8e84bbdbc4c6081e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:43 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:48.413933 12222 raft_consensus.cc:2243] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:48.434661 12222 raft_consensus.cc:2272] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:48.438233 12222 ts_tablet_manager.cc:1916] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:48.441509 11554 tablet_bootstrap.cc:492] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:48.441927 11554 ts_tablet_manager.cc:1403] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.268s user 0.022s sys 0.043s
I20260809 01:45:48.444211 11554 raft_consensus.cc:359] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.444871 11554 raft_consensus.cc:385] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:48.445132 11554 raft_consensus.cc:740] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:48.447037 11951 ts_tablet_manager.cc:1950] T 12fb852b8ea8478cb160a265459865c4 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:48.450925 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 168) took 6571 ms. Trace:
I20260809 01:45:48.451212 11951 rpcz_store.cc:276] 0809 01:45:41.879325 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.148828 (+269503us) service_pool.cc:224] Handling call
0809 01:45:48.082032 (+5933204us) ts_tablet_manager.cc:1151] Deleting tablet 12fb852b8ea8478cb160a265459865c4
0809 01:45:48.082079 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.214122 (+132043us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.424261 (+210139us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.450859 (+ 26598us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5697788}
I20260809 01:45:48.453439 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719
I20260809 01:45:48.453308 11554 consensus_queue.cc:260] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.454882 10597 catalog_manager.cc:6807] ReplaceTablet: tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 deleted and replaced by tablet 6542a2b55ce94835a31c732552d87222
I20260809 01:45:48.454924 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 166) took 6577 ms. Trace:
I20260809 01:45:48.455205 11951 rpcz_store.cc:276] 0809 01:45:41.878451 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.146435 (+267984us) service_pool.cc:224] Handling call
0809 01:45:48.454570 (+6308135us) ts_tablet_manager.cc:1151] Deleting tablet c528b45de0d14c19af5d6836b7a14f1c
0809 01:45:48.454619 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.454876 (+ 257us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5997486}
I20260809 01:45:48.456631 11110 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 with delete_type TABLET_DATA_DELETED (replaced by tablet 6542a2b55ce94835a31c732552d87222) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:48.457379 12386 raft_consensus.cc:493] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:48.457835 12386 raft_consensus.cc:515] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.459755 12386 leader_election.cc:290] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:48.465881 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389407e7ff464e0d8e84bbdbc4c6081e" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:48.466464 11346 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 with delete_type TABLET_DATA_DELETED (replaced by tablet 6542a2b55ce94835a31c732552d87222) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:48.466516 11156 raft_consensus.cc:2468] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:48.467597 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 12fb852b8ea8478cb160a265459865c4 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]) successfully deleted
I20260809 01:45:48.467581 11283 leader_election.cc:304] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
W20260809 01:45:48.468257 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet c528b45de0d14c19af5d6836b7a14f1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c528b45de0d14c19af5d6836b7a14f1c already in progress: creating tablet
I20260809 01:45:48.468293 12386 raft_consensus.cc:2804] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:48.468606 12386 raft_consensus.cc:493] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:48.468925 12386 raft_consensus.cc:3060] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:48.471915 11554 ts_tablet_manager.cc:1434] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.030s user 0.003s sys 0.003s
I20260809 01:45:48.474296 11554 tablet_bootstrap.cc:492] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:48.475834 11951 tablet_replica.cc:333] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:48.481326 11951 raft_consensus.cc:2243] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
W20260809 01:45:48.482077 10580 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet f1d5fdaaed9b40ceab053797439a38c9 (table default.Table_4f9017aa8338441e84bc6913c4915e42 [id=1c059f33595f4ec3a5fa72d085282765], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet f1d5fdaaed9b40ceab053797439a38c9 has no consensus state
I20260809 01:45:48.483198 11951 raft_consensus.cc:2272] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:48.485589 12386 raft_consensus.cc:515] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.488020 11554 tablet_bootstrap.cc:654] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:48.492624 11951 ts_tablet_manager.cc:1916] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:48.493106 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389407e7ff464e0d8e84bbdbc4c6081e" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:48.494127 11156 raft_consensus.cc:3060] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:48.497722 10588 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 9b51f999fab24b1ca7451ff543a124a4
I20260809 01:45:48.501473 11346 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b51f999fab24b1ca7451ff543a124a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:48 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:48.501875 11554 tablet_bootstrap.cc:492] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:48.507532 12299 raft_consensus.cc:493] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:48.508064 12299 raft_consensus.cc:515] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.502389 11554 ts_tablet_manager.cc:1403] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.028s user 0.018s sys 0.004s
I20260809 01:45:48.510100 12299 leader_election.cc:290] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:48.510804 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7df6343423604a00a46f203f61e98e2f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:48.511982 11951 ts_tablet_manager.cc:1929] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
W20260809 01:45:48.512152 11283 leader_election.cc:343] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:48.512336 11951 log.cc:1199] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/2aac67e7819f4a70b1050b558868f27f
I20260809 01:45:48.513065 11951 ts_tablet_manager.cc:1950] T 2aac67e7819f4a70b1050b558868f27f P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:48.515213 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 167) took 6636 ms. Trace:
I20260809 01:45:48.515793 11551 tablet_bootstrap.cc:654] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:48.515547 11951 rpcz_store.cc:276] 0809 01:45:41.879017 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.187329 (+308312us) service_pool.cc:224] Handling call
0809 01:45:48.472784 (+6285455us) ts_tablet_manager.cc:1151] Deleting tablet 2aac67e7819f4a70b1050b558868f27f
0809 01:45:48.475716 (+ 2932us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.502207 (+ 26491us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.511860 (+ 9653us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.515151 (+ 3291us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4506,"tablet-delete.queue_time_us":6005876}
I20260809 01:45:48.518344 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 173) took 6375 ms. Trace:
I20260809 01:45:48.518661 11951 rpcz_store.cc:276] 0809 01:45:42.142839 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.489595 (+346756us) service_pool.cc:224] Handling call
0809 01:45:48.517777 (+6028182us) ts_tablet_manager.cc:1151] Deleting tablet c16897857df24c6da5744a99d5f687c0
0809 01:45:48.517919 (+ 142us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.518293 (+ 374us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6027623}
I20260809 01:45:48.520454 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 174) took 6377 ms. Trace:
I20260809 01:45:48.520787 11951 rpcz_store.cc:276] 0809 01:45:42.143081 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.490230 (+347149us) service_pool.cc:224] Handling call
0809 01:45:48.520063 (+6029833us) ts_tablet_manager.cc:1151] Deleting tablet b2726cb09f734ed18bb9379f94f3f22a
0809 01:45:48.520118 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.520401 (+ 283us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6029067}
I20260809 01:45:48.522415 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 180) took 6105 ms. Trace:
I20260809 01:45:48.522478 11554 raft_consensus.cc:359] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.522730 11951 rpcz_store.cc:276] 0809 01:45:42.417089 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.928386 (+511297us) service_pool.cc:224] Handling call
0809 01:45:48.522043 (+5593657us) ts_tablet_manager.cc:1151] Deleting tablet 91941616dfdd4df3aaa7c8460876d273
0809 01:45:48.522091 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.522367 (+ 276us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5592837}
I20260809 01:45:48.523116 11554 raft_consensus.cc:385] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:48.523379 11554 raft_consensus.cc:740] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:48.524257 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 181) took 5644 ms. Trace:
I20260809 01:45:48.524169 11554 consensus_queue.cc:260] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.524546 11951 rpcz_store.cc:276] 0809 01:45:42.880425 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.943485 (+ 63060us) service_pool.cc:224] Handling call
0809 01:45:48.523870 (+5580385us) ts_tablet_manager.cc:1151] Deleting tablet 197b5043f6d348f89853c073b256daa4
0809 01:45:48.523920 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.524211 (+ 291us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5579509}
I20260809 01:45:48.526522 12386 leader_election.cc:290] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:48.527263 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 182) took 5645 ms. Trace:
I20260809 01:45:48.527822 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 2aac67e7819f4a70b1050b558868f27f (table default.Table_72e975bfbe594052a24baee87785c71a [id=04d4b772004b44e781c255cf271fab34]) successfully deleted
I20260809 01:45:48.527993 11951 rpcz_store.cc:276] 0809 01:45:42.882093 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.944481 (+ 62388us) service_pool.cc:224] Handling call
0809 01:45:48.526885 (+5582404us) ts_tablet_manager.cc:1151] Deleting tablet cd09bea5263344919ce26593fe958547
0809 01:45:48.526944 (+ 59us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.527218 (+ 274us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5581385}
I20260809 01:45:48.528479 11554 ts_tablet_manager.cc:1434] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.019s user 0.005s sys 0.001s
W20260809 01:45:48.528486 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet c16897857df24c6da5744a99d5f687c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c16897857df24c6da5744a99d5f687c0 already in progress: creating tablet
W20260809 01:45:48.529088 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet b2726cb09f734ed18bb9379f94f3f22a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b2726cb09f734ed18bb9379f94f3f22a already in progress: creating tablet
W20260809 01:45:48.529666 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 91941616dfdd4df3aaa7c8460876d273 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91941616dfdd4df3aaa7c8460876d273 already in progress: creating tablet
W20260809 01:45:48.530218 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 197b5043f6d348f89853c073b256daa4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 197b5043f6d348f89853c073b256daa4 already in progress: creating tablet
W20260809 01:45:48.530766 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet cd09bea5263344919ce26593fe958547 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd09bea5263344919ce26593fe958547 already in progress: creating tablet
I20260809 01:45:48.532614 11554 tablet_bootstrap.cc:492] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:48.535331 11383 raft_consensus.cc:2468] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:48.546777 11156 raft_consensus.cc:2468] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:48.552506 11124 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b51f999fab24b1ca7451ff543a124a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:48 UTC) from {username='slave'} at 127.0.0.1:56566
W20260809 01:45:48.554438 10599 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c704d148da8d406aa824253a51b850c8 consensus state has no leader
I20260809 01:45:48.556788 12222 ts_tablet_manager.cc:1929] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:48.557128 12222 log.cc:1199] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/637f0d6261f041b5aeff5fcc709a3dfc
I20260809 01:45:48.557852 12222 ts_tablet_manager.cc:1950] T 637f0d6261f041b5aeff5fcc709a3dfc P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:48.560195 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 214) took 1912 ms. Trace:
I20260809 01:45:48.560493 12222 rpcz_store.cc:276] 0809 01:45:46.647941 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.649155 (+ 1214us) service_pool.cc:224] Handling call
0809 01:45:48.298348 (+1649193us) ts_tablet_manager.cc:1151] Deleting tablet 637f0d6261f041b5aeff5fcc709a3dfc
0809 01:45:48.298403 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.434415 (+136012us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b5400110654. stack: 00007fe4bc3e2cf3 00007fe4bc3e2bf0 00007fe4bbf90574 0000000000516de4 0000000000516d79 00007fe4c34e3951 00007fe4c3582742 00007fe4c39c2d2c 00007fe4c462a41a 00007fe4c465b0e0 00007fe4c464164d 00007fe4c46415f9 00007fe4c4641589 00007fe4c4641551 00007fe4c464077d 00000000005bd8c4
0809 01:45:48.446892 (+ 12477us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.556669 (+109777us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.560143 (+ 3474us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3790,"spinlock_wait_cycles":81198464,"tablet-delete.queue_time_us":1648584}
I20260809 01:45:48.561977 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 217) took 1827 ms. Trace:
I20260809 01:45:48.562453 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 637f0d6261f041b5aeff5fcc709a3dfc (table default.Table_50f4db524f554499a2dab6d3bc48f053 [id=85ee30d1d77d47f691e6a0dd498d9bca]) successfully deleted
I20260809 01:45:48.578224 10858 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 with delete_type TABLET_DATA_DELETED (replaced by tablet 6542a2b55ce94835a31c732552d87222) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:48.562270 12222 rpcz_store.cc:276] 0809 01:45:46.734464 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.749675 (+ 15211us) service_pool.cc:224] Handling call
0809 01:45:48.561667 (+1811992us) ts_tablet_manager.cc:1151] Deleting tablet 5af35cf31a66474cb0575b2fb1547616
0809 01:45:48.561705 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.561932 (+ 227us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1811276}
W20260809 01:45:48.581789 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 5af35cf31a66474cb0575b2fb1547616 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5af35cf31a66474cb0575b2fb1547616 already in progress: creating tablet
I20260809 01:45:48.581995 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 216) took 1848 ms. Trace:
I20260809 01:45:48.582337 12222 rpcz_store.cc:276] 0809 01:45:46.733775 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.787073 (+ 53298us) service_pool.cc:224] Handling call
0809 01:45:48.581144 (+1794071us) ts_tablet_manager.cc:1151] Deleting tablet b84f1943c1a34e999fc7c12a5847cdb6
0809 01:45:48.581647 (+ 503us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.581933 (+ 286us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1793249}
I20260809 01:45:48.583375 12222 tablet_replica.cc:333] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:48.584028 12222 raft_consensus.cc:2243] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:48.584501 12222 raft_consensus.cc:2272] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:48.584652 10864 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b51f999fab24b1ca7451ff543a124a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:48 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:48.586268 12222 ts_tablet_manager.cc:1916] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:48.586378 10877 tablet_service.cc:1511] Processing CreateTablet for tablet d146bdaf4dd2418ca870c4afeb6ccb6c (DEFAULT_TABLE table=default.Table_62a39e76a73049dea65f8948c5513aa1 [id=4070cd0e58944571831808565b658bbe]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:48.586616 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet b84f1943c1a34e999fc7c12a5847cdb6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b84f1943c1a34e999fc7c12a5847cdb6 already in progress: creating tablet
I20260809 01:45:48.588212 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d146bdaf4dd2418ca870c4afeb6ccb6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:48.600826 11553 raft_consensus.cc:359] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.606236 11553 raft_consensus.cc:385] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:48.606595 11553 raft_consensus.cc:740] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:48.608108 11553 consensus_queue.cc:260] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.626192 11347 tablet_service.cc:1558] Processing DeleteTablet for tablet 12fb852b8ea8478cb160a265459865c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 7706838afc9f4dbebc3213b82c4da126) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:48.539269 11554 tablet_bootstrap.cc:654] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:48.627278 11062 leader_election.cc:304] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
W20260809 01:45:48.627936 11062 raft_consensus.cc:2686] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 01:45:48.539880 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 195) took 5481 ms. Trace:
I20260809 01:45:48.549497 11283 leader_election.cc:304] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
W20260809 01:45:48.630206 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 80784.2us
I20260809 01:45:48.630290 12386 raft_consensus.cc:2804] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:48.630726 12386 raft_consensus.cc:697] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:48.629276 11951 rpcz_store.cc:276] 0809 01:45:43.058541 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.953909 (+895368us) service_pool.cc:224] Handling call
0809 01:45:48.529114 (+4575205us) ts_tablet_manager.cc:1151] Deleting tablet b3b02bd8c93b43bf90ff0192a9541aae
0809 01:45:48.539523 (+ 10409us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f19b52 00007f0eeae871d9 00007f0eea09f4e8 00007f0eea165aa9 00007f0eea1660a3 00007f0eea1970e0 00007f0eea17d64d 00007f0eea17d5f9 00007f0eea17d589
0809 01:45:48.539567 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.539832 (+ 265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4887296,"tablet-delete.queue_time_us":4574633}
I20260809 01:45:48.631490 12386 consensus_queue.cc:237] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.632356 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 196) took 5441 ms. Trace:
I20260809 01:45:48.632622 11951 rpcz_store.cc:276] 0809 01:45:43.190644 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.954538 (+763894us) service_pool.cc:224] Handling call
0809 01:45:48.631955 (+4677417us) ts_tablet_manager.cc:1151] Deleting tablet 761370ec8b164fd184a2e091030f610b
0809 01:45:48.632014 (+ 59us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.632296 (+ 282us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4676648}
I20260809 01:45:48.635608 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 197) took 5429 ms. Trace:
I20260809 01:45:48.635951 11951 rpcz_store.cc:276] 0809 01:45:43.206564 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.955383 (+748819us) service_pool.cc:224] Handling call
0809 01:45:48.633390 (+4678007us) ts_tablet_manager.cc:1151] Deleting tablet bd3662c307ba4ff78b8f3165b23acb1a
0809 01:45:48.633447 (+ 57us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.635549 (+ 2102us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4677301}
I20260809 01:45:48.636984 11951 tablet_replica.cc:333] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:48.638485 11951 raft_consensus.cc:2243] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:48.639138 11951 raft_consensus.cc:2272] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:48.642129 11951 ts_tablet_manager.cc:1916] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:48.647764 12482 raft_consensus.cc:493] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:48.648491 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet b3b02bd8c93b43bf90ff0192a9541aae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b3b02bd8c93b43bf90ff0192a9541aae already in progress: creating tablet
I20260809 01:45:48.648326 12482 raft_consensus.cc:515] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:48.651460 10861 consensus_meta.cc:287] T 00d27eecce7447eda6771b7bd0196193 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent flushing consensus metadata: real 0.527s user 0.004s sys 0.015s
I20260809 01:45:48.651708 12482 leader_election.cc:290] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:48.652325 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 761370ec8b164fd184a2e091030f610b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 761370ec8b164fd184a2e091030f610b already in progress: creating tablet
I20260809 01:45:48.652415 12482 raft_consensus.cc:493] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:48.652972 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet bd3662c307ba4ff78b8f3165b23acb1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd3662c307ba4ff78b8f3165b23acb1a already in progress: creating tablet
I20260809 01:45:48.653007 12482 raft_consensus.cc:515] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:48.654966 12482 leader_election.cc:290] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:48.655440 12482 raft_consensus.cc:493] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:48.655961 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f022e6a57a4ec0a5a8c33159e97a8e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:48.658804 10804 leader_election.cc:343] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:48.669831 12482 raft_consensus.cc:515] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.671226 11554 tablet_bootstrap.cc:492] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:48.671962 11554 ts_tablet_manager.cc:1403] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.139s user 0.015s sys 0.001s
I20260809 01:45:48.679359 12222 ts_tablet_manager.cc:1929] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:48.679704 12222 log.cc:1199] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/7501d985958746bea58e0ff752d7fed8
I20260809 01:45:48.680279 12222 ts_tablet_manager.cc:1950] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:48.680630 12482 leader_election.cc:290] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:48.673624 11949 ts_tablet_manager.cc:1929] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:48.681321 11949 log.cc:1199] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/7803a6aff02d42c1833f9b2cd7332b9f
I20260809 01:45:48.681627 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38bd344108649ed8e1687f82154000b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:48.682366 11949 ts_tablet_manager.cc:1950] T 7803a6aff02d42c1833f9b2cd7332b9f P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:48.682794 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb9afd3be194830ba99f5e20fdaca0a" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:48.683032 10804 leader_election.cc:343] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:48.683221 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 218) took 4694 ms. Trace:
W20260809 01:45:48.684087 10804 leader_election.cc:343] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:48.674352 11553 ts_tablet_manager.cc:1434] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.443s user 0.008s sys 0.004s
I20260809 01:45:48.684176 11554 raft_consensus.cc:359] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.684823 11554 raft_consensus.cc:385] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:48.685076 11554 raft_consensus.cc:740] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:48.685334 11553 tablet_bootstrap.cc:492] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:48.690397 11554 consensus_queue.cc:260] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.691145 12299 raft_consensus.cc:493] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:48.691531 12299 raft_consensus.cc:515] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.692040 12556 raft_consensus.cc:493] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:48.692497 12556 raft_consensus.cc:515] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.694633 12556 leader_election.cc:290] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:48.696507 12299 leader_election.cc:290] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:48.699667 11554 ts_tablet_manager.cc:1434] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.027s user 0.006s sys 0.001s
I20260809 01:45:48.704497 11554 tablet_bootstrap.cc:492] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:48.705319 10589 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_a58e460fc9644019b0d51c55b3673fea" } modify_external_catalogs: true
I20260809 01:45:48.705909 10589 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_a58e460fc9644019b0d51c55b3673fea" } modify_external_catalogs: true
I20260809 01:45:48.678653 12549 raft_consensus.cc:493] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:48.710278 12549 raft_consensus.cc:515] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.712359 12549 leader_election.cc:290] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:48.714983 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38bd344108649ed8e1687f82154000b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:48.717185 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb9afd3be194830ba99f5e20fdaca0a" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:48.717684 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9493892c1ff48e7a0558ff0fcb7a0f3" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:48.718068 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b51f999fab24b1ca7451ff543a124a4" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:48.719354 10804 leader_election.cc:343] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:48.719352 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3684fa56f9549f492a96a5c765eda78" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:48.719365 11283 leader_election.cc:343] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:48.720464 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3684fa56f9549f492a96a5c765eda78" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:48.721740 10804 leader_election.cc:343] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:48.722033 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9493892c1ff48e7a0558ff0fcb7a0f3" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:48.722455 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 218) took 1885 ms. Trace:
I20260809 01:45:48.722786 12222 rpcz_store.cc:276] 0809 01:45:46.836994 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.837142 (+ 148us) service_pool.cc:224] Handling call
0809 01:45:48.583231 (+1746089us) ts_tablet_manager.cc:1151] Deleting tablet 7501d985958746bea58e0ff752d7fed8
0809 01:45:48.583270 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.622406 (+ 39136us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.679233 (+ 56827us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.722394 (+ 43161us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1744892}
I20260809 01:45:48.724049 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 7501d985958746bea58e0ff752d7fed8 (table default.Table_03cfb94f08614dc5bcdaf9eb182adc1b [id=d23f238a284a47f2be2faf340b853440]) successfully deleted
I20260809 01:45:48.724792 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f022e6a57a4ec0a5a8c33159e97a8e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:48.683578 11949 rpcz_store.cc:276] 0809 01:45:43.989151 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.989283 (+ 132us) service_pool.cc:224] Handling call
0809 01:45:48.039564 (+4050281us) ts_tablet_manager.cc:1151] Deleting tablet 7803a6aff02d42c1833f9b2cd7332b9f
0809 01:45:48.039605 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.120196 (+ 80591us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.669366 (+549170us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.683146 (+ 13780us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35712,"tablet-delete.queue_time_us":4049516}
W20260809 01:45:48.726616 11063 leader_election.cc:343] T 7501d985958746bea58e0ff752d7fed8 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 7501d985958746bea58e0ff752d7fed8
I20260809 01:45:48.727169 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 219) took 3436 ms. Trace:
I20260809 01:45:48.727959 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 7803a6aff02d42c1833f9b2cd7332b9f (table default.Table_8a405d19ab94416c8d58fff878edae0a [id=42039f2e75f942ac8b96fb47088ad193]) successfully deleted
I20260809 01:45:48.727487 11949 rpcz_store.cc:276] 0809 01:45:45.290803 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.296698 (+ 5895us) service_pool.cc:224] Handling call
0809 01:45:48.726801 (+3430103us) ts_tablet_manager.cc:1151] Deleting tablet b2726cb09f734ed18bb9379f94f3f22a
0809 01:45:48.726903 (+ 102us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.727103 (+ 200us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3429239}
W20260809 01:45:48.729875 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet b2726cb09f734ed18bb9379f94f3f22a because the tablet was not found. No further retry: Not found: Tablet not found: b2726cb09f734ed18bb9379f94f3f22a
I20260809 01:45:48.730209 11949 tablet_replica.cc:333] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:48.735693 11551 tablet_bootstrap.cc:492] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:48.736176 11551 ts_tablet_manager.cc:1403] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.361s user 0.013s sys 0.002s
I20260809 01:45:48.736253 12222 tablet_replica.cc:333] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:48.737509 12222 raft_consensus.cc:2243] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:48.743772 11949 raft_consensus.cc:2243] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:48.744299 11949 raft_consensus.cc:2272] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:48.738651 11551 raft_consensus.cc:359] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.745144 11551 raft_consensus.cc:385] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:48.745617 11551 raft_consensus.cc:740] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:48.745493 12299 consensus_queue.cc:1048] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:48.744388 12222 raft_consensus.cc:2272] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:48.747736 12222 ts_tablet_manager.cc:1916] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:48.761179 12556 raft_consensus.cc:493] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:48.761742 12556 raft_consensus.cc:515] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.748354 11551 consensus_queue.cc:260] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.766655 11551 ts_tablet_manager.cc:1434] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260809 01:45:48.768507 10597 master_service.cc:960] ReplaceTablet: received request to replace tablet 81da8d08131d4a3ba636ecfc7a7d2951 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:48.770511 11551 tablet_bootstrap.cc:492] T 59de95452c0a4945beae5a9344537e0f P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:48.773831 12549 raft_consensus.cc:493] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:48.774490 12549 raft_consensus.cc:515] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.777891 11554 tablet_bootstrap.cc:654] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:48.784795 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_b254da32a96842368d87fa797c32915a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:48.786463 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b254da32a96842368d87fa797c32915a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:48.790562 12299 raft_consensus.cc:493] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:48.790992 12299 raft_consensus.cc:515] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.792870 12299 leader_election.cc:290] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:48.796705 11551 tablet_bootstrap.cc:654] T 59de95452c0a4945beae5a9344537e0f P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:48.796665 11283 leader_election.cc:343] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: 8f0d79e03e66410487a72f1b47b327a0
I20260809 01:45:48.784924 11951 ts_tablet_manager.cc:1929] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:48.797322 11951 log.cc:1199] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/ef8a334741d3419eaa377338e0eaa889
I20260809 01:45:48.798039 11951 ts_tablet_manager.cc:1950] T ef8a334741d3419eaa377338e0eaa889 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:48.806234 10591 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_5c934da8f7cf4e3f91d0964d5984d94d" } new_table_name: "default.Table_8d898b49c47e4d0591f1cc0dc2972d36" modify_external_catalogs: true
I20260809 01:45:48.814502 11949 ts_tablet_manager.cc:1916] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:48.815922 12556 leader_election.cc:290] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:48.816929 10595 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 7df6343423604a00a46f203f61e98e2f (table default.Table_18e8960a1b8a457fb622abd881d541ed [id=b7fa92e91f85479c82ba9d384c44d429]). Expected version 1 got 0
I20260809 01:45:48.818876 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8baa4ae62edf46b5a386661a6826bbab" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:48.822278 10804 leader_election.cc:343] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:48.827615 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_0ccb4fdb37aa49a2bf24e84e07a54081"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:48.826025 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 200) took 5558 ms. Trace:
W20260809 01:45:48.829147 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ccb4fdb37aa49a2bf24e84e07a54081 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:48.828683 11951 rpcz_store.cc:276] 0809 01:45:43.267789 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:43.972669 (+704880us) service_pool.cc:224] Handling call
0809 01:45:48.636796 (+4664127us) ts_tablet_manager.cc:1151] Deleting tablet ef8a334741d3419eaa377338e0eaa889
0809 01:45:48.636867 (+ 71us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.687126 (+ 50259us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.784754 (+ 97628us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.825958 (+ 41204us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4663339}
I20260809 01:45:48.829774 10588 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_aed078d7fcc9482aa4c52c4664e16dc2" } new_table_name: "default.Table_a21564a1b68c4bc9b78278dfa70ad1a9" modify_external_catalogs: true
I20260809 01:45:48.830665 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet ef8a334741d3419eaa377338e0eaa889 (table default.Table_52926e76c3ad4bbcbc425c32d4473a3c [id=51771c6c664942e4b1df1132b5641ffa]) successfully deleted
I20260809 01:45:48.837891 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_1d10d5edc68f414f89a9b12b052c7aae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:48.838754 10596 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet d146bdaf4dd2418ca870c4afeb6ccb6c
W20260809 01:45:48.839296 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1d10d5edc68f414f89a9b12b052c7aae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:48.839473 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 205) took 5140 ms. Trace:
I20260809 01:45:48.839910 11951 rpcz_store.cc:276] 0809 01:45:43.698727 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.097592 (+398865us) service_pool.cc:224] Handling call
0809 01:45:48.838918 (+4741326us) ts_tablet_manager.cc:1151] Deleting tablet c3bf43978dc94c2b9a7f3c18a4677c40
0809 01:45:48.839005 (+ 87us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.839389 (+ 384us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4740697}
I20260809 01:45:48.843305 11554 tablet_bootstrap.cc:492] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:48.843713 11554 ts_tablet_manager.cc:1403] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.143s user 0.011s sys 0.000s
I20260809 01:45:48.843971 11118 tablet_service.cc:1558] Processing DeleteTablet for tablet d146bdaf4dd2418ca870c4afeb6ccb6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:48 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:48.844496 12222 ts_tablet_manager.cc:1929] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260809 01:45:48.845099 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet d146bdaf4dd2418ca870c4afeb6ccb6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:48 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:48.846218 11554 raft_consensus.cc:359] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.846813 11554 raft_consensus.cc:385] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:48.847061 11554 raft_consensus.cc:740] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:48.847421 11551 tablet_bootstrap.cc:492] T 59de95452c0a4945beae5a9344537e0f P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:48.847869 11551 ts_tablet_manager.cc:1403] T 59de95452c0a4945beae5a9344537e0f P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.078s user 0.011s sys 0.000s
I20260809 01:45:48.850423 11551 raft_consensus.cc:359] T 59de95452c0a4945beae5a9344537e0f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.851127 12549 leader_election.cc:290] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:48.851567 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet c3bf43978dc94c2b9a7f3c18a4677c40 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c3bf43978dc94c2b9a7f3c18a4677c40 already in progress: creating tablet
I20260809 01:45:48.851919 11347 tablet_service.cc:1558] Processing DeleteTablet for tablet d146bdaf4dd2418ca870c4afeb6ccb6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:48 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:48.852525 11551 raft_consensus.cc:385] T 59de95452c0a4945beae5a9344537e0f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:48.855110 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7df6343423604a00a46f203f61e98e2f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:48.855839 10910 raft_consensus.cc:2468] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:48.860057 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 206) took 5065 ms. Trace:
I20260809 01:45:48.860337 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b51f999fab24b1ca7451ff543a124a4" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:48.860903 10908 raft_consensus.cc:2468] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:48.860544 11951 rpcz_store.cc:276] 0809 01:45:43.794687 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.098319 (+303632us) service_pool.cc:224] Handling call
0809 01:45:48.859487 (+4761168us) ts_tablet_manager.cc:1151] Deleting tablet 403cb09c1786491ab0d40502038bf9c0
0809 01:45:48.859572 (+ 85us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.859973 (+ 401us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4760522}
I20260809 01:45:48.861646 11551 raft_consensus.cc:740] T 59de95452c0a4945beae5a9344537e0f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:48.852893 12222 log.cc:1199] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/ead75d766d0346f6823074d90ba8054e
I20260809 01:45:48.852838 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cef88ffe9e95437da81793795742def3" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:48.857725 11118 tablet_service.cc:1558] Processing DeleteTablet for tablet ef8a334741d3419eaa377338e0eaa889 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f771c8051784da18a6de9fcf45e0125) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:48.862666 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0d79e03e66410487a72f1b47b327a0" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:48.863445 12222 ts_tablet_manager.cc:1950] T ead75d766d0346f6823074d90ba8054e P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:48.863826 10908 raft_consensus.cc:2468] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
W20260809 01:45:48.864856 10804 leader_election.cc:343] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:48.865504 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 219) took 1656 ms. Trace:
I20260809 01:45:48.865720 11554 consensus_queue.cc:260] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.865831 12222 rpcz_store.cc:276] 0809 01:45:47.208647 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:47.208786 (+ 139us) service_pool.cc:224] Handling call
0809 01:45:48.736104 (+1527318us) ts_tablet_manager.cc:1151] Deleting tablet ead75d766d0346f6823074d90ba8054e
0809 01:45:48.736146 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.793187 (+ 57041us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.844370 (+ 51183us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.865445 (+ 21075us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1526719}
W20260809 01:45:48.867554 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 403cb09c1786491ab0d40502038bf9c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 403cb09c1786491ab0d40502038bf9c0 already in progress: creating tablet
I20260809 01:45:48.861922 11951 tablet_replica.cc:333] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:48.870429 11951 raft_consensus.cc:2243] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:48.873767 11551 consensus_queue.cc:260] T 59de95452c0a4945beae5a9344537e0f P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.874454 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet ead75d766d0346f6823074d90ba8054e (table default.Table_3c356f5b797347c1ab98aaca3d06f4eb [id=028331846d434fc1beff5b11e2129233]) successfully deleted
I20260809 01:45:48.875907 11551 ts_tablet_manager.cc:1434] T 59de95452c0a4945beae5a9344537e0f P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.028s user 0.005s sys 0.000s
I20260809 01:45:48.876711 11551 tablet_bootstrap.cc:492] T 7df6343423604a00a46f203f61e98e2f P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:48.880061 11951 pending_rounds.cc:70] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316: Trying to abort 1 pending ops.
I20260809 01:45:48.880324 11951 pending_rounds.cc:77] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438816832450560 op_type: NO_OP noop_request { }
I20260809 01:45:48.880681 11951 raft_consensus.cc:2889] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:48.880993 11951 raft_consensus.cc:2272] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:48.881667 11554 ts_tablet_manager.cc:1434] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.038s user 0.008s sys 0.000s
I20260809 01:45:48.882107 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 232) took 1099 ms. Trace:
I20260809 01:45:48.882629 11554 tablet_bootstrap.cc:492] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:48.882440 12222 rpcz_store.cc:276] 0809 01:45:47.782579 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:47.782754 (+ 175us) service_pool.cc:224] Handling call
0809 01:45:48.881737 (+1098983us) ts_tablet_manager.cc:1151] Deleting tablet e93334cc766044e79d37f293bc8556fb
0809 01:45:48.881779 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.882056 (+ 277us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1097857}
W20260809 01:45:48.883618 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet e93334cc766044e79d37f293bc8556fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e93334cc766044e79d37f293bc8556fb already in progress: creating tablet
I20260809 01:45:48.884617 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 230) took 1113 ms. Trace:
I20260809 01:45:48.884933 12222 rpcz_store.cc:276] 0809 01:45:47.771645 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:47.771782 (+ 137us) service_pool.cc:224] Handling call
0809 01:45:48.884291 (+1112509us) ts_tablet_manager.cc:1151] Deleting tablet 977694496e014ef29f5d7d513e5219c0
0809 01:45:48.884340 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.884567 (+ 227us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1099492}
W20260809 01:45:48.886242 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 977694496e014ef29f5d7d513e5219c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 977694496e014ef29f5d7d513e5219c0 already in progress: creating tablet
I20260809 01:45:48.890687 11951 ts_tablet_manager.cc:1916] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:48.894073 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 231) took 1121 ms. Trace:
I20260809 01:45:48.894369 12222 rpcz_store.cc:276] 0809 01:45:47.772726 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:47.777669 (+ 4943us) service_pool.cc:224] Handling call
0809 01:45:48.893755 (+1116086us) ts_tablet_manager.cc:1151] Deleting tablet 19fe81e277cd4c798a07ceafa0b60566
0809 01:45:48.893796 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.894022 (+ 226us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1108200}
I20260809 01:45:48.895088 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 81da8d08131d4a3ba636ecfc7a7d2951
I20260809 01:45:48.895728 11551 tablet_bootstrap.cc:654] T 7df6343423604a00a46f203f61e98e2f P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:48.896973 10597 catalog_manager.cc:6807] ReplaceTablet: tablet 81da8d08131d4a3ba636ecfc7a7d2951 deleted and replaced by tablet cb13f4c7b07348cba67b6b43a9b154bd
I20260809 01:45:48.897301 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet 81da8d08131d4a3ba636ecfc7a7d2951 with delete_type TABLET_DATA_DELETED (replaced by tablet cb13f4c7b07348cba67b6b43a9b154bd) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:48.896517 11553 tablet_bootstrap.cc:700] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent opening tablet: real 0.119s user 0.002s sys 0.000s
I20260809 01:45:48.898845 11553 tablet_bootstrap.cc:654] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:48.898926 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 19fe81e277cd4c798a07ceafa0b60566 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 19fe81e277cd4c798a07ceafa0b60566 already in progress: creating tablet
I20260809 01:45:48.899351 12549 raft_consensus.cc:493] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:48.899603 11118 tablet_service.cc:1558] Processing DeleteTablet for tablet 81da8d08131d4a3ba636ecfc7a7d2951 with delete_type TABLET_DATA_DELETED (replaced by tablet cb13f4c7b07348cba67b6b43a9b154bd) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:48.899833 12549 raft_consensus.cc:515] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.904244 12222 tablet_replica.cc:333] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
W20260809 01:45:48.904407 10591 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9a354a7cc2a84123a456125ef6c9828a has no consensus state
I20260809 01:45:48.904523 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8baa4ae62edf46b5a386661a6826bbab" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:48.906292 12222 raft_consensus.cc:2243] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:48.906633 11347 tablet_service.cc:1558] Processing DeleteTablet for tablet 81da8d08131d4a3ba636ecfc7a7d2951 with delete_type TABLET_DATA_DELETED (replaced by tablet cb13f4c7b07348cba67b6b43a9b154bd) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:48.906771 12222 raft_consensus.cc:2272] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:48.908505 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389407e7ff464e0d8e84bbdbc4c6081e" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:48.908947 10591 master_service.cc:960] ReplaceTablet: received request to replace tablet bb7a8b6d06064addb02979a7feedf756 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:48.910327 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389407e7ff464e0d8e84bbdbc4c6081e" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:48.910887 11554 tablet_bootstrap.cc:654] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:48.914831 12222 ts_tablet_manager.cc:1916] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:48.916290 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_5f7c40d82c854aaa87deac85836396ca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:48.918010 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f7c40d82c854aaa87deac85836396ca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:48.919416 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cef88ffe9e95437da81793795742def3" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:48.920972 11289 leader_election.cc:304] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:48.921880 11551 tablet_bootstrap.cc:492] T 7df6343423604a00a46f203f61e98e2f P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:48.922010 12299 raft_consensus.cc:2804] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:48.922266 11551 ts_tablet_manager.cc:1403] T 7df6343423604a00a46f203f61e98e2f P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.046s user 0.009s sys 0.000s
I20260809 01:45:48.922358 12299 raft_consensus.cc:493] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:48.922657 12299 raft_consensus.cc:3060] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:48.924609 11551 raft_consensus.cc:359] T 7df6343423604a00a46f203f61e98e2f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.925225 11551 raft_consensus.cc:385] T 7df6343423604a00a46f203f61e98e2f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:48.929936 12299 raft_consensus.cc:515] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.932633 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7df6343423604a00a46f203f61e98e2f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:48.934623 12299 leader_election.cc:290] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:48.925472 11551 raft_consensus.cc:740] T 7df6343423604a00a46f203f61e98e2f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:48.938274 11551 consensus_queue.cc:260] T 7df6343423604a00a46f203f61e98e2f P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:48.938966 11289 consensus_peers.cc:597] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 135d3d65ea7846d3bc2cb59d44dbedf6. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:48.939021 11156 raft_consensus.cc:3060] T 7df6343423604a00a46f203f61e98e2f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:48.939873 11289 leader_election.cc:304] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:48.941045 12386 raft_consensus.cc:2804] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:48.941360 12386 raft_consensus.cc:493] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:48.941684 12386 raft_consensus.cc:3060] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:48.945464 11156 raft_consensus.cc:2468] T 7df6343423604a00a46f203f61e98e2f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:48.948832 11283 leader_election.cc:304] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:48.950906 12386 raft_consensus.cc:515] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.952342 11551 ts_tablet_manager.cc:1434] T 7df6343423604a00a46f203f61e98e2f P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260809 01:45:48.954938 12386 leader_election.cc:290] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:48.955521 12386 raft_consensus.cc:2804] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:48.955991 12386 raft_consensus.cc:697] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:48.956812 12386 consensus_queue.cc:237] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:48.958916 11551 tablet_bootstrap.cc:492] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
W20260809 01:45:48.960518 10588 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c704d148da8d406aa824253a51b850c8 consensus state has no leader
I20260809 01:45:48.961648 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b51f999fab24b1ca7451ff543a124a4" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
W20260809 01:45:48.971359 10805 leader_election.cc:343] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:48.971967 10805 leader_election.cc:304] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:48.972643 12556 raft_consensus.cc:2749] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:48.976311 11551 tablet_bootstrap.cc:654] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:48.987663 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f45c02a6994c5aaaf9c0a499e15fed" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:48.989022 10804 leader_election.cc:343] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:48.990432 12222 ts_tablet_manager.cc:1929] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:48.990744 12222 log.cc:1199] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/135d3d65ea7846d3bc2cb59d44dbedf6
I20260809 01:45:48.991544 12549 leader_election.cc:290] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:48.994047 10805 leader_election.cc:343] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:48.994458 10805 leader_election.cc:304] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:48.995450 10805 leader_election.cc:343] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:48.995814 10805 leader_election.cc:304] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:48.999895 12222 ts_tablet_manager.cc:1950] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:48.962028 11951 ts_tablet_manager.cc:1929] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:49.001722 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 233) took 1111 ms. Trace:
I20260809 01:45:49.001870 11951 log.cc:1199] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/da86b5e4a27544c59ae6bb387440d246
I20260809 01:45:49.002008 12222 rpcz_store.cc:276] 0809 01:45:47.890001 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:47.890140 (+ 139us) service_pool.cc:224] Handling call
0809 01:45:48.904110 (+1013970us) ts_tablet_manager.cc:1151] Deleting tablet 135d3d65ea7846d3bc2cb59d44dbedf6
0809 01:45:48.904148 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.930230 (+ 26082us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.990322 (+ 60092us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.001665 (+ 11343us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1013398}
I20260809 01:45:49.002661 11951 ts_tablet_manager.cc:1950] T da86b5e4a27544c59ae6bb387440d246 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:49.004077 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 135d3d65ea7846d3bc2cb59d44dbedf6 (table default.Table_53624396796f4684acbc3787e6e05f6e [id=18e87d9e1fa3442aaa489e3abcd4640e]) successfully deleted
I20260809 01:45:49.004673 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 207) took 4115 ms. Trace:
I20260809 01:45:49.004971 11951 rpcz_store.cc:276] 0809 01:45:44.889215 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:44.889446 (+ 231us) service_pool.cc:224] Handling call
0809 01:45:48.861775 (+3972329us) ts_tablet_manager.cc:1151] Deleting tablet da86b5e4a27544c59ae6bb387440d246
0809 01:45:48.861821 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.927091 (+ 65270us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:48.961901 (+ 34810us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.004614 (+ 42713us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8775,"tablet-delete.queue_time_us":3971498}
W20260809 01:45:48.962823 11283 leader_election.cc:343] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:48.966163 11554 tablet_bootstrap.cc:492] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:49.006450 11554 ts_tablet_manager.cc:1403] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.124s user 0.015s sys 0.000s
I20260809 01:45:49.008709 11554 raft_consensus.cc:359] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.009294 11554 raft_consensus.cc:385] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:48.970194 11289 leader_election.cc:304] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:49.009613 11553 tablet_bootstrap.cc:492] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:49.010263 11553 ts_tablet_manager.cc:1403] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.325s user 0.009s sys 0.002s
I20260809 01:45:49.010847 11554 raft_consensus.cc:740] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.011849 11554 consensus_queue.cc:260] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.015563 12222 tablet_replica.cc:333] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:49.016254 12222 raft_consensus.cc:2243] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:49.014159 11951 tablet_replica.cc:333] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:49.017896 11951 raft_consensus.cc:2243] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:49.018640 11951 raft_consensus.cc:2272] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:49.016774 12222 raft_consensus.cc:2272] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:49.020725 11951 ts_tablet_manager.cc:1916] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:49.021073 12222 ts_tablet_manager.cc:1916] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:49.021237 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 750aa437ef594587a512a9c9c8278aa8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 750aa437ef594587a512a9c9c8278aa8 already in progress: creating tablet
I20260809 01:45:49.021727 12556 raft_consensus.cc:2749] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.021778 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet da86b5e4a27544c59ae6bb387440d246 (table default.Table_29a4bd6bd10a473dbd9dbda008b9eb16 [id=67ee87fabc7749279341528fbcce16d4]) successfully deleted
I20260809 01:45:49.022415 12556 raft_consensus.cc:2749] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.022475 12299 raft_consensus.cc:2804] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:49.022800 12299 raft_consensus.cc:493] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:49.023102 12299 raft_consensus.cc:3060] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:49.023656 10805 leader_election.cc:343] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.024035 10805 leader_election.cc:304] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:49.025029 10805 leader_election.cc:343] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.025369 10805 leader_election.cc:304] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:49.026068 12556 raft_consensus.cc:2749] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.026600 12556 raft_consensus.cc:2749] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.030213 11553 raft_consensus.cc:359] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.031606 11554 ts_tablet_manager.cc:1434] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.025s user 0.006s sys 0.000s
I20260809 01:45:49.031865 11156 raft_consensus.cc:3060] T 59de95452c0a4945beae5a9344537e0f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:49.050190 12579 consensus_queue.cc:1048] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:49.051263 11553 raft_consensus.cc:385] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.051621 11553 raft_consensus.cc:740] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.051728 11551 tablet_bootstrap.cc:492] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:49.052165 11551 ts_tablet_manager.cc:1403] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.093s user 0.013s sys 0.000s
I20260809 01:45:49.052860 10597 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_056eba06842f4044b4a9f9922d208c59" } modify_external_catalogs: true
I20260809 01:45:49.053380 10597 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_056eba06842f4044b4a9f9922d208c59" } modify_external_catalogs: true
W20260809 01:45:49.054416 10805 leader_election.cc:343] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.061678 10805 leader_election.cc:304] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:49.061718 11551 raft_consensus.cc:359] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.063104 11554 tablet_bootstrap.cc:492] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:49.063792 12545 raft_consensus.cc:2749] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:49.065376 10805 leader_election.cc:343] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.072103 10805 leader_election.cc:304] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:49.072889 12545 raft_consensus.cc:2749] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.072824 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_cc3003c3373b4f6cbcae8eb85030466f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:49.074616 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cc3003c3373b4f6cbcae8eb85030466f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:49.074705 11289 leader_election.cc:343] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Illegal state: Tablet not RUNNING: STOPPED
I20260809 01:45:49.057407 11949 ts_tablet_manager.cc:1929] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:49.077934 10591 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet bb7a8b6d06064addb02979a7feedf756
I20260809 01:45:49.053424 11553 consensus_queue.cc:260] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.079367 10591 catalog_manager.cc:6807] ReplaceTablet: tablet bb7a8b6d06064addb02979a7feedf756 deleted and replaced by tablet 2126670a2f29408a9ba4b949f966bf76
I20260809 01:45:49.079831 11347 tablet_service.cc:1558] Processing DeleteTablet for tablet bb7a8b6d06064addb02979a7feedf756 with delete_type TABLET_DATA_DELETED (replaced by tablet 2126670a2f29408a9ba4b949f966bf76) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:49.082649 10877 tablet_service.cc:1558] Processing DeleteTablet for tablet bb7a8b6d06064addb02979a7feedf756 with delete_type TABLET_DATA_DELETED (replaced by tablet 2126670a2f29408a9ba4b949f966bf76) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:49.082870 11118 tablet_service.cc:1558] Processing DeleteTablet for tablet bb7a8b6d06064addb02979a7feedf756 with delete_type TABLET_DATA_DELETED (replaced by tablet 2126670a2f29408a9ba4b949f966bf76) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:49.062383 11551 raft_consensus.cc:385] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.083590 11551 raft_consensus.cc:740] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
W20260809 01:45:49.084551 11283 consensus_peers.cc:597] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 637f0d6261f041b5aeff5fcc709a3dfc. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:49.084206 11551 consensus_queue.cc:260] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.090710 11553 ts_tablet_manager.cc:1434] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.080s user 0.006s sys 0.000s
I20260809 01:45:49.091117 11551 ts_tablet_manager.cc:1434] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.039s user 0.007s sys 0.000s
I20260809 01:45:49.091946 11551 tablet_bootstrap.cc:492] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:49.094465 11553 tablet_bootstrap.cc:492] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:49.102381 12299 raft_consensus.cc:515] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.069293 11554 tablet_bootstrap.cc:654] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.078110 11949 log.cc:1199] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/389407e7ff464e0d8e84bbdbc4c6081e
W20260809 01:45:49.107457 11283 leader_election.cc:343] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: 8f0d79e03e66410487a72f1b47b327a0
W20260809 01:45:49.107546 11289 leader_election.cc:343] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Illegal state: Tablet not RUNNING: STOPPED
I20260809 01:45:49.108806 11949 ts_tablet_manager.cc:1950] T 389407e7ff464e0d8e84bbdbc4c6081e P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:49.109473 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_8edd49df5953499184f2a39b02c69e91" } new_table_name: "default.Table_e7e9cb467df0423e9e1cc43a4d87bfd8" modify_external_catalogs: true
I20260809 01:45:49.110814 12299 leader_election.cc:290] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:49.115958 11551 tablet_bootstrap.cc:654] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.116992 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f45c02a6994c5aaaf9c0a499e15fed" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:49.124105 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 229) took 3456 ms. Trace:
I20260809 01:45:49.124747 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7df6343423604a00a46f203f61e98e2f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:49.128031 10910 raft_consensus.cc:3060] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:49.125725 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b51f999fab24b1ca7451ff543a124a4" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:49.133927 10905 raft_consensus.cc:3060] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:49.136958 12222 ts_tablet_manager.cc:1929] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20260809 01:45:49.140395 11554 tablet_bootstrap.cc:492] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:49.141000 11554 ts_tablet_manager.cc:1403] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.078s user 0.010s sys 0.004s
I20260809 01:45:49.143991 11951 ts_tablet_manager.cc:1929] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:49.144523 11951 log.cc:1199] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/7501d985958746bea58e0ff752d7fed8
I20260809 01:45:49.145406 11951 ts_tablet_manager.cc:1950] T 7501d985958746bea58e0ff752d7fed8 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:49.146464 10905 raft_consensus.cc:2468] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:49.137475 12222 log.cc:1199] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/72b4850de7d740d794283ca93152fdc1
I20260809 01:45:49.150741 12222 ts_tablet_manager.cc:1950] T 72b4850de7d740d794283ca93152fdc1 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:49.151300 12545 raft_consensus.cc:493] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.151723 12545 raft_consensus.cc:515] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.153236 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 72b4850de7d740d794283ca93152fdc1 (table default.Table_ff0d273341b54d79b4e634ea694e982f [id=46b096a453084568935bc0d5855cb0f7]) successfully deleted
I20260809 01:45:49.154773 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e03033672e947eab8cc143263281133" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:49.154670 11554 raft_consensus.cc:359] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.154855 12545 leader_election.cc:290] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:49.155290 11554 raft_consensus.cc:385] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.155570 11554 raft_consensus.cc:740] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
W20260809 01:45:49.156075 10804 leader_election.cc:343] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.140457 10910 raft_consensus.cc:2468] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:49.156395 11554 consensus_queue.cc:260] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.147171 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 208) took 4134 ms. Trace:
I20260809 01:45:49.157232 11951 rpcz_store.cc:276] 0809 01:45:45.012735 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.013067 (+ 332us) service_pool.cc:224] Handling call
0809 01:45:49.014013 (+4000946us) ts_tablet_manager.cc:1151] Deleting tablet 7501d985958746bea58e0ff752d7fed8
0809 01:45:49.014064 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:49.121560 (+107496us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.143815 (+ 22255us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.147116 (+ 3301us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3668327}
I20260809 01:45:49.157150 11949 rpcz_store.cc:276] 0809 01:45:45.667601 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.667721 (+ 120us) service_pool.cc:224] Handling call
0809 01:45:48.730039 (+3062318us) ts_tablet_manager.cc:1151] Deleting tablet 389407e7ff464e0d8e84bbdbc4c6081e
0809 01:45:48.730092 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:48.997120 (+267028us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.057284 (+ 60164us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.124051 (+ 66767us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12769,"tablet-delete.queue_time_us":3028201}
I20260809 01:45:49.158648 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 7501d985958746bea58e0ff752d7fed8 (table default.Table_03cfb94f08614dc5bcdaf9eb182adc1b [id=d23f238a284a47f2be2faf340b853440]) successfully deleted
I20260809 01:45:49.158713 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 389407e7ff464e0d8e84bbdbc4c6081e (table default.Table_47a7750fcf1c4cf688f25aff5ded233d [id=1fb02cbd4f2d4807b326a8fff08316d8]) successfully deleted
I20260809 01:45:49.159735 11949 tablet_replica.cc:333] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:49.160646 11949 raft_consensus.cc:2243] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:49.160787 11554 ts_tablet_manager.cc:1434] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.019s user 0.002s sys 0.006s
I20260809 01:45:49.161255 11949 raft_consensus.cc:2272] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:49.162969 12222 tablet_replica.cc:333] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:49.163570 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0d79e03e66410487a72f1b47b327a0" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:49.163707 12222 raft_consensus.cc:2243] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:49.164127 10908 raft_consensus.cc:3060] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:49.164157 12222 raft_consensus.cc:2272] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:49.165972 11551 tablet_bootstrap.cc:492] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:49.166333 11551 ts_tablet_manager.cc:1403] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.075s user 0.009s sys 0.000s
I20260809 01:45:49.167946 12299 raft_consensus.cc:493] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.168354 12299 raft_consensus.cc:515] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.168623 11551 raft_consensus.cc:359] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.169214 11551 raft_consensus.cc:385] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.169449 11551 raft_consensus.cc:740] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.170431 12299 leader_election.cc:290] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:49.170336 12583 consensus_queue.cc:1048] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:49.174022 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af35cf31a66474cb0575b2fb1547616" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:49.174046 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e03033672e947eab8cc143263281133" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:49.174551 11289 leader_election.cc:304] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:49.174938 11551 consensus_queue.cc:260] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:49.176210 11283 leader_election.cc:343] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.176808 12222 ts_tablet_manager.cc:1916] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:49.176782 12583 raft_consensus.cc:2804] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:49.177206 12583 raft_consensus.cc:697] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:49.177999 12583 consensus_queue.cc:237] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.178707 10908 raft_consensus.cc:2468] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:49.183346 11551 ts_tablet_manager.cc:1434] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20260809 01:45:49.187279 11551 tablet_bootstrap.cc:492] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:49.192596 11551 tablet_bootstrap.cc:654] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.192961 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 218) took 3477 ms. Trace:
I20260809 01:45:49.193293 11951 rpcz_store.cc:276] 0809 01:45:45.715709 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.757850 (+ 42141us) service_pool.cc:224] Handling call
0809 01:45:49.161186 (+3403336us) ts_tablet_manager.cc:1151] Deleting tablet 50f45c02a6994c5aaaf9c0a499e15fed
0809 01:45:49.192618 (+ 31432us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:49.192906 (+ 288us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3401742}
W20260809 01:45:49.194550 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 50f45c02a6994c5aaaf9c0a499e15fed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50f45c02a6994c5aaaf9c0a499e15fed already in progress: creating tablet
I20260809 01:45:49.190367 11554 tablet_bootstrap.cc:492] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:49.195436 12527 raft_consensus.cc:493] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.195880 12527 raft_consensus.cc:515] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.197880 12527 leader_election.cc:290] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:49.199873 11062 leader_election.cc:343] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Not found: Tablet not found: ef8a334741d3419eaa377338e0eaa889
I20260809 01:45:49.200205 11951 tablet_replica.cc:333] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:49.200896 11951 raft_consensus.cc:2243] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
W20260809 01:45:49.201116 10805 leader_election.cc:343] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:49.201364 11063 leader_election.cc:343] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: ef8a334741d3419eaa377338e0eaa889
I20260809 01:45:49.201570 11951 pending_rounds.cc:70] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316: Trying to abort 1 pending ops.
I20260809 01:45:49.201778 11951 pending_rounds.cc:77] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438829602803712 op_type: NO_OP noop_request { }
I20260809 01:45:49.201733 11063 leader_election.cc:304] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:49.201766 10805 leader_election.cc:304] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:49.202095 11951 raft_consensus.cc:2889] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:49.202448 11951 raft_consensus.cc:2272] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:49.202576 10805 raft_consensus.cc:2686] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 01:45:49.202651 12527 raft_consensus.cc:2749] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:49.204457 10805 leader_election.cc:343] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.204873 10805 leader_election.cc:304] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:49.215916 11951 ts_tablet_manager.cc:1916] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:49.220757 10598 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_01317dd35d864bfbae02d0d3a30e31cd" } new_table_name: "default.Table_ec33f983dcf94fbab30efdd0aa4596c6" modify_external_catalogs: true
I20260809 01:45:49.223052 11553 tablet_bootstrap.cc:654] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.224658 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_b2acb056eb544b64b517a5b0fdd3f131"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:49.226241 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b2acb056eb544b64b517a5b0fdd3f131 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:49.226451 11554 tablet_bootstrap.cc:654] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.230137 10588 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_9f368169db0b40ee91c03bdc0b8d4785"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:49.231549 10588 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f368169db0b40ee91c03bdc0b8d4785 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:49.237267 10586 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8a02b2e4506b41d995d66a6e35f9b050 consensus state has no leader
I20260809 01:45:49.241995 11949 ts_tablet_manager.cc:1916] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:49.244179 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_779631fdff574e96846c0dd384b81c9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:49.245608 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_779631fdff574e96846c0dd384b81c9f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:49.246963 12527 raft_consensus.cc:493] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.247406 12527 raft_consensus.cc:515] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:49.250432 10598 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a1e8ce18ca6246b3b89ef824b75cefab has no consensus state
I20260809 01:45:49.250612 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4754f2e2a35f4a6a91fa333e2cf564e4" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:49.251195 11383 raft_consensus.cc:2468] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:49.253202 12545 raft_consensus.cc:2749] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.253856 12545 raft_consensus.cc:493] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.254266 12545 raft_consensus.cc:515] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.254988 11062 leader_election.cc:304] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:49.249341 12527 leader_election.cc:290] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:49.256564 12527 raft_consensus.cc:2804] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:49.256882 12527 raft_consensus.cc:493] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:49.257157 12527 raft_consensus.cc:3060] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:49.260589 12545 leader_election.cc:290] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:49.261122 11951 ts_tablet_manager.cc:1929] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260809 01:45:49.261777 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 57340.5us
I20260809 01:45:49.261765 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6159d660d544b3e907a08372027c0e2" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:49.262701 11289 leader_election.cc:304] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:49.263471 12583 raft_consensus.cc:2804] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:49.263926 12583 raft_consensus.cc:697] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
W20260809 01:45:49.264169 11063 leader_election.cc:343] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 4754f2e2a35f4a6a91fa333e2cf564e4
I20260809 01:45:49.264698 12583 consensus_queue.cc:237] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:49.265086 10804 leader_election.cc:343] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.265638 12545 raft_consensus.cc:493] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.266093 12545 raft_consensus.cc:515] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.268662 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7a511512184f53921cea081f82aa37" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:49.268852 12545 leader_election.cc:290] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:49.269881 10804 leader_election.cc:343] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.277817 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6159d660d544b3e907a08372027c0e2" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:49.279865 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af35cf31a66474cb0575b2fb1547616" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:49.280572 10908 raft_consensus.cc:2468] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:49.261498 11951 log.cc:1199] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/389407e7ff464e0d8e84bbdbc4c6081e
I20260809 01:45:49.292004 11951 ts_tablet_manager.cc:1950] T 389407e7ff464e0d8e84bbdbc4c6081e P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:49.294049 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 217) took 3581 ms. Trace:
I20260809 01:45:49.294391 11951 rpcz_store.cc:276] 0809 01:45:45.713706 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.753783 (+ 40077us) service_pool.cc:224] Handling call
0809 01:45:49.200005 (+3446222us) ts_tablet_manager.cc:1151] Deleting tablet 389407e7ff464e0d8e84bbdbc4c6081e
0809 01:45:49.200050 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:49.238739 (+ 38689us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.261001 (+ 22262us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.293970 (+ 32969us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3399087}
I20260809 01:45:49.295554 11553 tablet_bootstrap.cc:492] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:49.295907 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 389407e7ff464e0d8e84bbdbc4c6081e (table default.Table_47a7750fcf1c4cf688f25aff5ded233d [id=1fb02cbd4f2d4807b326a8fff08316d8]) successfully deleted
I20260809 01:45:49.296841 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7a511512184f53921cea081f82aa37" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:49.298506 11289 leader_election.cc:304] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:49.299497 12583 raft_consensus.cc:2804] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:49.299866 12583 raft_consensus.cc:493] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:49.300235 12583 raft_consensus.cc:3060] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:49.302809 11551 tablet_bootstrap.cc:492] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:49.303218 11551 ts_tablet_manager.cc:1403] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.116s user 0.005s sys 0.005s
W20260809 01:45:49.306901 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 52029.4us
I20260809 01:45:49.307156 12583 raft_consensus.cc:515] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.307665 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 219) took 3462 ms. Trace:
I20260809 01:45:49.284456 12527 raft_consensus.cc:515] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.307994 11951 rpcz_store.cc:276] 0809 01:45:45.844785 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.844979 (+ 194us) service_pool.cc:224] Handling call
0809 01:45:49.307115 (+3462136us) ts_tablet_manager.cc:1151] Deleting tablet 8323a7a40a1445f980c0adcf6a6d16c7
0809 01:45:49.307377 (+ 262us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:49.307612 (+ 235us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3442951}
W20260809 01:45:49.309183 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 8323a7a40a1445f980c0adcf6a6d16c7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8323a7a40a1445f980c0adcf6a6d16c7 already in progress: creating tablet
I20260809 01:45:49.309888 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af35cf31a66474cb0575b2fb1547616" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
W20260809 01:45:49.311177 11283 leader_election.cc:343] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.311532 12527 leader_election.cc:290] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:49.313705 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4754f2e2a35f4a6a91fa333e2cf564e4" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:49.314242 11383 raft_consensus.cc:3060] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:49.314419 12222 ts_tablet_manager.cc:1929] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:49.312785 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 220) took 3452 ms. Trace:
I20260809 01:45:49.315485 11951 rpcz_store.cc:276] 0809 01:45:45.860174 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.865066 (+ 4892us) service_pool.cc:224] Handling call
0809 01:45:49.311694 (+3446628us) ts_tablet_manager.cc:1151] Deleting tablet 66cc1253dee24f5d858d521ddbbfb620
0809 01:45:49.311736 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:49.312733 (+ 997us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3401513}
I20260809 01:45:49.315333 12222 log.cc:1199] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/389407e7ff464e0d8e84bbdbc4c6081e
I20260809 01:45:49.316748 12222 ts_tablet_manager.cc:1950] T 389407e7ff464e0d8e84bbdbc4c6081e P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
W20260809 01:45:49.316674 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 66cc1253dee24f5d858d521ddbbfb620 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66cc1253dee24f5d858d521ddbbfb620 already in progress: creating tablet
I20260809 01:45:49.318358 11951 tablet_replica.cc:333] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
W20260809 01:45:49.321969 11283 consensus_peers.cc:597] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ead75d766d0346f6823074d90ba8054e. This is attempt 1: this message will repeat every 5th retry.
W20260809 01:45:49.318336 11063 leader_election.cc:343] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 4754f2e2a35f4a6a91fa333e2cf564e4
I20260809 01:45:49.319072 11951 raft_consensus.cc:2243] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:49.323637 11951 raft_consensus.cc:2272] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:49.317205 11551 raft_consensus.cc:359] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.325088 11383 raft_consensus.cc:2468] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:49.325366 11951 ts_tablet_manager.cc:1916] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:49.326377 11553 ts_tablet_manager.cc:1403] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.232s user 0.010s sys 0.000s
W20260809 01:45:49.310099 10805 leader_election.cc:343] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.327910 10805 leader_election.cc:304] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:49.328634 12545 raft_consensus.cc:2749] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.328936 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 389407e7ff464e0d8e84bbdbc4c6081e (table default.Table_47a7750fcf1c4cf688f25aff5ded233d [id=1fb02cbd4f2d4807b326a8fff08316d8]) successfully deleted
I20260809 01:45:49.326260 11062 leader_election.cc:304] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:49.330107 12545 raft_consensus.cc:493] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.330231 12527 raft_consensus.cc:2804] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:49.330513 12545 raft_consensus.cc:515] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.331770 11554 tablet_bootstrap.cc:492] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:49.332160 11554 ts_tablet_manager.cc:1403] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.142s user 0.010s sys 0.018s
I20260809 01:45:49.333055 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e4315b3c644846ac572883c51f23de" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:49.333261 12545 leader_election.cc:290] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:49.334198 12527 raft_consensus.cc:697] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
W20260809 01:45:49.334321 10804 leader_election.cc:343] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:49.334941 10805 leader_election.cc:343] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.334673 11551 raft_consensus.cc:385] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.335001 11554 raft_consensus.cc:359] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.335316 10805 leader_election.cc:304] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:49.335606 11554 raft_consensus.cc:385] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.335386 11551 raft_consensus.cc:740] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.335877 11554 raft_consensus.cc:740] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.337721 12545 raft_consensus.cc:2749] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.339287 11553 raft_consensus.cc:359] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.339887 11553 raft_consensus.cc:385] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.340147 11553 raft_consensus.cc:740] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.340467 12527 consensus_queue.cc:237] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.340673 11554 consensus_queue.cc:260] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.341941 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e4315b3c644846ac572883c51f23de" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:49.343564 12545 raft_consensus.cc:493] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.344040 12545 raft_consensus.cc:515] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:49.345335 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 already in progress: creating tablet
I20260809 01:45:49.346658 11554 ts_tablet_manager.cc:1434] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.014s user 0.007s sys 0.001s
I20260809 01:45:49.346628 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2692a931500443d298e25a2b362b22a1" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:49.346823 12545 leader_election.cc:290] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:49.347934 10804 leader_election.cc:343] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:49.348222 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 9b51f999fab24b1ca7451ff543a124a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b51f999fab24b1ca7451ff543a124a4 already in progress: creating tablet
W20260809 01:45:49.349830 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet d146bdaf4dd2418ca870c4afeb6ccb6c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d146bdaf4dd2418ca870c4afeb6ccb6c already in progress: creating tablet
I20260809 01:45:49.350611 12222 tablet_replica.cc:333] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:49.351603 12222 raft_consensus.cc:2243] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:49.352058 12222 raft_consensus.cc:2272] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:49.341492 11551 consensus_queue.cc:260] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.353215 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2692a931500443d298e25a2b362b22a1" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:49.353781 11554 tablet_bootstrap.cc:492] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:49.354282 11551 ts_tablet_manager.cc:1434] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.051s user 0.008s sys 0.000s
W20260809 01:45:49.354671 10805 leader_election.cc:343] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.355078 10805 leader_election.cc:304] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:49.355540 11551 tablet_bootstrap.cc:492] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
W20260809 01:45:49.356041 10805 leader_election.cc:343] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.356395 10805 leader_election.cc:304] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:49.357069 12549 raft_consensus.cc:2749] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.357712 12549 raft_consensus.cc:2749] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.359127 11554 tablet_bootstrap.cc:654] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.359601 11553 consensus_queue.cc:260] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.360477 12583 leader_election.cc:290] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:49.376232 12545 raft_consensus.cc:493] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.376681 12545 raft_consensus.cc:515] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.379413 12545 leader_election.cc:290] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:49.379995 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa12353c25cb4da680d478ce104bc6ba" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:49.380155 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_50689eb6dda841ebbb61bfbe0e21c1cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:49.381207 10804 leader_election.cc:343] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.384100 11553 ts_tablet_manager.cc:1434] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.057s user 0.005s sys 0.000s
W20260809 01:45:49.388867 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_50689eb6dda841ebbb61bfbe0e21c1cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:49.390846 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa12353c25cb4da680d478ce104bc6ba" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:49.391219 11554 tablet_bootstrap.cc:492] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:49.391633 11554 ts_tablet_manager.cc:1403] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.044s user 0.008s sys 0.001s
W20260809 01:45:49.394248 10805 leader_election.cc:343] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.394670 10805 leader_election.cc:304] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:49.381384 11951 ts_tablet_manager.cc:1929] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:49.395412 11951 log.cc:1199] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/637f0d6261f041b5aeff5fcc709a3dfc
I20260809 01:45:49.395620 12545 raft_consensus.cc:2749] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.396220 11951 ts_tablet_manager.cc:1950] T 637f0d6261f041b5aeff5fcc709a3dfc P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:49.396798 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af35cf31a66474cb0575b2fb1547616" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:49.394057 11554 raft_consensus.cc:359] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.397298 11554 raft_consensus.cc:385] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.397491 10908 raft_consensus.cc:3060] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:49.397586 11554 raft_consensus.cc:740] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.400480 11554 consensus_queue.cc:260] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.401639 12599 raft_consensus.cc:493] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.402093 12599 raft_consensus.cc:515] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.404769 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b51f999fab24b1ca7451ff543a124a4" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:49.405145 12599 leader_election.cc:290] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:49.405303 10908 raft_consensus.cc:2468] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:49.406627 11553 tablet_bootstrap.cc:492] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:49.407083 12222 ts_tablet_manager.cc:1916] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:49.405458 11554 ts_tablet_manager.cc:1434] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.014s user 0.003s sys 0.003s
I20260809 01:45:49.417991 11289 leader_election.cc:304] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:49.418918 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b51f999fab24b1ca7451ff543a124a4" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:49.419124 12583 raft_consensus.cc:2804] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:49.421360 10908 raft_consensus.cc:2393] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33b5652538e64ed58a09993d3188f8d7 in current term 1: Already voted for candidate 4f60c9ac9414403d9ea5a97b103c2316 in this term.
I20260809 01:45:49.425274 11063 leader_election.cc:304] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:49.419816 12583 raft_consensus.cc:697] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:49.426455 12527 raft_consensus.cc:3060] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:49.429811 12583 consensus_queue.cc:237] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.431491 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 225) took 3021 ms. Trace:
I20260809 01:45:49.431809 11951 rpcz_store.cc:276] 0809 01:45:46.409975 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.410094 (+ 119us) service_pool.cc:224] Handling call
0809 01:45:49.318210 (+2908116us) ts_tablet_manager.cc:1151] Deleting tablet 637f0d6261f041b5aeff5fcc709a3dfc
0809 01:45:49.318250 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:49.366563 (+ 48313us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.381257 (+ 14694us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.431428 (+ 50171us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2907602}
I20260809 01:45:49.432920 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 226) took 3021 ms. Trace:
I20260809 01:45:49.433190 11951 rpcz_store.cc:276] 0809 01:45:46.411336 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.411460 (+ 124us) service_pool.cc:224] Handling call
0809 01:45:49.432684 (+3021224us) ts_tablet_manager.cc:1151] Deleting tablet c795f81d2bc248eeb39b64d6aaa0370c
0809 01:45:49.432722 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:49.432871 (+ 149us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3020718}
I20260809 01:45:49.434171 11951 tablet_replica.cc:333] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:49.434897 11951 raft_consensus.cc:2243] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:49.433648 11551 tablet_bootstrap.cc:654] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.435535 11951 pending_rounds.cc:70] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316: Trying to abort 1 pending ops.
I20260809 01:45:49.435765 11951 pending_rounds.cc:77] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438832198307840 op_type: NO_OP noop_request { }
I20260809 01:45:49.436097 11951 raft_consensus.cc:2889] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:49.436411 11951 raft_consensus.cc:2272] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:49.436972 10588 master_service.cc:960] ReplaceTablet: received request to replace tablet a38bd344108649ed8e1687f82154000b from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:49.437554 12299 raft_consensus.cc:493] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.437958 12299 raft_consensus.cc:515] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.438915 12527 raft_consensus.cc:2749] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.439885 12299 leader_election.cc:290] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:49.441227 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16897857df24c6da5744a99d5f687c0" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:49.441574 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 637f0d6261f041b5aeff5fcc709a3dfc (table default.Table_50f4db524f554499a2dab6d3bc48f053 [id=85ee30d1d77d47f691e6a0dd498d9bca]) successfully deleted
W20260809 01:45:49.442221 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet c795f81d2bc248eeb39b64d6aaa0370c because the tablet was not found. No further retry: Not found: Tablet not found: c795f81d2bc248eeb39b64d6aaa0370c
I20260809 01:45:49.443082 10596 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_c36eea2ca6e040ba888f487a8e539e04" } new_table_name: "default.Table_7dd2cf9954234c83b63d37aebee151d5" modify_external_catalogs: true
W20260809 01:45:49.446076 11283 leader_election.cc:343] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.447333 11554 tablet_bootstrap.cc:492] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
W20260809 01:45:49.447963 11289 leader_election.cc:343] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: c16897857df24c6da5744a99d5f687c0
I20260809 01:45:49.448299 11289 leader_election.cc:304] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:49.450261 12583 raft_consensus.cc:2749] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.457883 11554 tablet_bootstrap.cc:654] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.458997 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_7222335f5bc84f3992e7574872dfa0de" } new_table_name: "default.Table_67bd4213ff9c478b9a51efdfe77bcdb7" modify_external_catalogs: true
I20260809 01:45:49.460567 11551 tablet_bootstrap.cc:492] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:49.461019 11551 ts_tablet_manager.cc:1403] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.106s user 0.010s sys 0.000s
I20260809 01:45:49.462497 11951 ts_tablet_manager.cc:1916] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:49.463431 11551 raft_consensus.cc:359] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.464020 11551 raft_consensus.cc:385] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.464315 11551 raft_consensus.cc:740] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.465157 11551 consensus_queue.cc:260] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.475298 11551 ts_tablet_manager.cc:1434] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.014s user 0.004s sys 0.004s
I20260809 01:45:49.480435 11551 tablet_bootstrap.cc:492] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
W20260809 01:45:49.482894 10596 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4aa1429f21024739b06bec88c3cb24d9 consensus state has no leader
I20260809 01:45:49.483099 11949 ts_tablet_manager.cc:1929] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:49.484670 11949 log.cc:1199] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/50f45c02a6994c5aaaf9c0a499e15fed
I20260809 01:45:49.486383 11551 tablet_bootstrap.cc:654] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.485400 11553 tablet_bootstrap.cc:654] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:49.490907 10586 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3938bfd21b294fa6a25f0d8efc1ca29a consensus state has no leader
I20260809 01:45:49.492565 11949 ts_tablet_manager.cc:1950] T 50f45c02a6994c5aaaf9c0a499e15fed P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:49.498809 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 230) took 3826 ms. Trace:
I20260809 01:45:49.499396 11554 tablet_bootstrap.cc:492] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:49.499148 11949 rpcz_store.cc:276] 0809 01:45:45.672363 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.672470 (+ 107us) service_pool.cc:224] Handling call
0809 01:45:49.159570 (+3487100us) ts_tablet_manager.cc:1151] Deleting tablet 50f45c02a6994c5aaaf9c0a499e15fed
0809 01:45:49.159626 (+ 56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:49.351910 (+192284us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.482939 (+131029us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.498759 (+ 15820us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3457478}
I20260809 01:45:49.499795 11554 ts_tablet_manager.cc:1403] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.053s user 0.007s sys 0.002s
I20260809 01:45:49.500396 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 50f45c02a6994c5aaaf9c0a499e15fed (table default.Table_eb6df7e248d44109898ab900441bda8b [id=7701d6a8eff047ddac4f8775385c57ae]) successfully deleted
I20260809 01:45:49.500761 11949 tablet_replica.cc:333] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:49.501585 11949 raft_consensus.cc:2243] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:49.502077 11949 raft_consensus.cc:2272] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:49.502539 11554 raft_consensus.cc:359] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.503343 11554 raft_consensus.cc:385] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.503687 11554 raft_consensus.cc:740] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.504187 11949 ts_tablet_manager.cc:1916] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:49.504462 11554 consensus_queue.cc:260] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:49.510303 10581 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 3938bfd21b294fa6a25f0d8efc1ca29a (table default.Table_67bd4213ff9c478b9a51efdfe77bcdb7 [id=01721069a381465397d92855a7accc23], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3938bfd21b294fa6a25f0d8efc1ca29a consensus state has no leader
I20260809 01:45:49.516577 11553 tablet_bootstrap.cc:492] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:49.517069 11553 ts_tablet_manager.cc:1403] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.111s user 0.007s sys 0.003s
I20260809 01:45:49.518663 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_1391d3ee4c884a43b3fa55ce4cf9d7d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:49.519773 11553 raft_consensus.cc:359] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:49.520123 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1391d3ee4c884a43b3fa55ce4cf9d7d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:49.520464 11553 raft_consensus.cc:385] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.520761 11553 raft_consensus.cc:740] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.521540 11553 consensus_queue.cc:260] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.521919 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_80fa389cbb634927981d6547e77d96fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:49.523360 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_80fa389cbb634927981d6547e77d96fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:49.524905 11553 ts_tablet_manager.cc:1434] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.008s user 0.002s sys 0.002s
I20260809 01:45:49.525996 11551 tablet_bootstrap.cc:492] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:49.526378 11551 ts_tablet_manager.cc:1403] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.046s user 0.013s sys 0.000s
I20260809 01:45:49.528730 11551 raft_consensus.cc:359] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.529363 11551 raft_consensus.cc:385] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.529722 11551 raft_consensus.cc:740] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.534451 10598 master_service.cc:960] ReplaceTablet: received request to replace tablet ebc209c696ac4c669961f5ed8392af40 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:49.535018 11553 tablet_bootstrap.cc:492] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:49.538987 12545 raft_consensus.cc:493] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.539637 12545 raft_consensus.cc:515] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.542924 12527 raft_consensus.cc:493] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.543499 12527 raft_consensus.cc:515] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.544346 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bc7dc3dfc2f4c1286ca24b8e3b20719" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:49.548009 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bc7dc3dfc2f4c1286ca24b8e3b20719" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:49.549544 10805 leader_election.cc:343] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.550273 12545 leader_election.cc:290] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:49.551143 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af35cf31a66474cb0575b2fb1547616" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:49.552403 10804 leader_election.cc:343] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.552757 10804 leader_election.cc:304] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:49.555168 12527 leader_election.cc:290] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:49.556806 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af35cf31a66474cb0575b2fb1547616" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:49.557554 10908 raft_consensus.cc:2393] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33b5652538e64ed58a09993d3188f8d7 in current term 1: Already voted for candidate 4f60c9ac9414403d9ea5a97b103c2316 in this term.
I20260809 01:45:49.558951 11063 leader_election.cc:304] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:49.560175 11553 tablet_bootstrap.cc:654] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.560137 12527 raft_consensus.cc:3060] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:49.563714 12545 raft_consensus.cc:2749] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.570199 11951 ts_tablet_manager.cc:1929] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:49.570609 11951 log.cc:1199] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/8f0d79e03e66410487a72f1b47b327a0
I20260809 01:45:49.571468 11951 ts_tablet_manager.cc:1950] T 8f0d79e03e66410487a72f1b47b327a0 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:49.571973 12222 ts_tablet_manager.cc:1929] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:49.572386 12222 log.cc:1199] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/ef8a334741d3419eaa377338e0eaa889
I20260809 01:45:49.562963 11554 ts_tablet_manager.cc:1434] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.063s user 0.007s sys 0.000s
I20260809 01:45:49.574777 11551 consensus_queue.cc:260] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.575680 12599 raft_consensus.cc:493] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.576217 12599 raft_consensus.cc:515] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.577183 12222 ts_tablet_manager.cc:1950] T ef8a334741d3419eaa377338e0eaa889 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:49.584077 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc230f25baf94cbf80aa8e2771ec23ba" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:49.584725 11383 raft_consensus.cc:2468] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:49.585732 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_19f955b06eb24b51815571c6ae99966b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:49.586382 11062 leader_election.cc:304] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
W20260809 01:45:49.590022 11063 leader_election.cc:343] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: dc230f25baf94cbf80aa8e2771ec23ba
I20260809 01:45:49.590888 12599 leader_election.cc:290] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:49.591917 12599 raft_consensus.cc:2804] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:49.592579 11554 tablet_bootstrap.cc:492] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:49.593061 12599 raft_consensus.cc:493] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:49.594404 12599 raft_consensus.cc:3060] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:49.580150 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet ef8a334741d3419eaa377338e0eaa889 (table default.Table_52926e76c3ad4bbcbc425c32d4473a3c [id=51771c6c664942e4b1df1132b5641ffa]) successfully deleted
W20260809 01:45:49.606838 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 81da8d08131d4a3ba636ecfc7a7d2951 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81da8d08131d4a3ba636ecfc7a7d2951 already in progress: creating tablet
W20260809 01:45:49.607566 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet bb7a8b6d06064addb02979a7feedf756 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb7a8b6d06064addb02979a7feedf756 already in progress: creating tablet
W20260809 01:45:49.608721 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_19f955b06eb24b51815571c6ae99966b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:49.611264 12527 raft_consensus.cc:2749] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.613845 11553 tablet_bootstrap.cc:492] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:49.614251 11553 ts_tablet_manager.cc:1403] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.080s user 0.014s sys 0.004s
I20260809 01:45:49.618930 11553 raft_consensus.cc:359] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.619534 11553 raft_consensus.cc:385] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.619804 11553 raft_consensus.cc:740] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.619998 12599 raft_consensus.cc:515] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.620549 11553 consensus_queue.cc:260] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.628762 11551 ts_tablet_manager.cc:1434] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.102s user 0.009s sys 0.000s
I20260809 01:45:49.631117 11551 tablet_bootstrap.cc:492] T cef88ffe9e95437da81793795742def3 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
W20260809 01:45:49.641587 11063 leader_election.cc:343] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: dc230f25baf94cbf80aa8e2771ec23ba
I20260809 01:45:49.629235 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 227) took 3153 ms. Trace:
I20260809 01:45:49.636700 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc230f25baf94cbf80aa8e2771ec23ba" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:49.646960 12599 leader_election.cc:290] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:49.647348 11383 raft_consensus.cc:3060] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:49.637621 12583 raft_consensus.cc:493] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.653815 12583 raft_consensus.cc:515] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.655722 11551 tablet_bootstrap.cc:654] T cef88ffe9e95437da81793795742def3 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:49.656070 10804 consensus_peers.cc:597] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 72b4850de7d740d794283ca93152fdc1. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:49.657207 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5050a590fe0240ac8c9f9191ec2475a1" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:49.658015 11553 ts_tablet_manager.cc:1434] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.044s user 0.007s sys 0.000s
W20260809 01:45:49.658578 11283 leader_election.cc:343] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.659214 11553 tablet_bootstrap.cc:492] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:49.660561 12583 leader_election.cc:290] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:49.662379 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_d5bd0ddd910f400b88147ea461999b39"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:49.663408 11289 leader_election.cc:343] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 5050a590fe0240ac8c9f9191ec2475a1
I20260809 01:45:49.663167 11951 rpcz_store.cc:276] 0809 01:45:46.475744 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.481622 (+ 5878us) service_pool.cc:224] Handling call
0809 01:45:49.434023 (+2952401us) ts_tablet_manager.cc:1151] Deleting tablet 8f0d79e03e66410487a72f1b47b327a0
0809 01:45:49.434062 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:49.505954 (+ 71892us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.570077 (+ 64123us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.629173 (+ 59096us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2951603}
I20260809 01:45:49.663800 11289 leader_election.cc:304] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:49.664434 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 8f0d79e03e66410487a72f1b47b327a0 (table default.Table_30d779db722c451baacdc0202550d5b6 [id=869fc1e71c7142c5bbd1fa358c94e453]) successfully deleted
W20260809 01:45:49.665333 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d5bd0ddd910f400b88147ea461999b39 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:49.665431 12583 raft_consensus.cc:2749] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.666586 11951 tablet_replica.cc:333] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:49.667383 11951 raft_consensus.cc:2243] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:49.667907 11951 raft_consensus.cc:2272] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:49.671196 11383 raft_consensus.cc:2468] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:49.672982 11062 leader_election.cc:304] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:49.674671 12599 raft_consensus.cc:2804] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:49.675233 12599 raft_consensus.cc:697] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:49.676214 12599 consensus_queue.cc:237] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:49.679247 10581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111) [suppressed 3 similar messages]
I20260809 01:45:49.681077 11949 ts_tablet_manager.cc:1929] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:49.681434 11949 log.cc:1199] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/8323a7a40a1445f980c0adcf6a6d16c7
I20260809 01:45:49.682129 11949 ts_tablet_manager.cc:1950] T 8323a7a40a1445f980c0adcf6a6d16c7 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:49.685268 11951 ts_tablet_manager.cc:1916] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:49.687352 11554 tablet_bootstrap.cc:654] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.687628 10596 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_1ac0a8ad913b419188050971d1b6bf6b" } new_table_name: "default.Table_f9ca1c8245e643c994d68dad6b152b09" modify_external_catalogs: true
I20260809 01:45:49.696769 11551 tablet_bootstrap.cc:492] T cef88ffe9e95437da81793795742def3 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:49.697126 11551 ts_tablet_manager.cc:1403] T cef88ffe9e95437da81793795742def3 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.068s user 0.006s sys 0.009s
I20260809 01:45:49.697196 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_ec33f983dcf94fbab30efdd0aa4596c6" } new_table_name: "default.Table_6bc9094f4ac34c81b10b00bca15191f1" modify_external_catalogs: true
W20260809 01:45:49.707477 11283 consensus_peers.cc:597] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 135d3d65ea7846d3bc2cb59d44dbedf6. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:49.701018 11551 raft_consensus.cc:359] T cef88ffe9e95437da81793795742def3 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.708019 11551 raft_consensus.cc:385] T cef88ffe9e95437da81793795742def3 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.708259 11551 raft_consensus.cc:740] T cef88ffe9e95437da81793795742def3 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.708894 11551 consensus_queue.cc:260] T cef88ffe9e95437da81793795742def3 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.713261 11551 ts_tablet_manager.cc:1434] T cef88ffe9e95437da81793795742def3 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.016s user 0.005s sys 0.003s
I20260809 01:45:49.714452 11551 tablet_bootstrap.cc:492] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:49.725775 12599 raft_consensus.cc:493] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.726449 12599 raft_consensus.cc:515] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.732179 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d1ed215ba644abbae09e78016b2c21" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:49.734114 11062 leader_election.cc:343] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: Tablet not RUNNING: STOPPED
I20260809 01:45:49.734367 12545 raft_consensus.cc:493] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.735045 12545 raft_consensus.cc:515] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:49.736620 10581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.125:42063, user_credentials={real_user=slave}} blocked reactor thread for 57371.5us
W20260809 01:45:49.739331 11063 leader_election.cc:343] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: a9d1ed215ba644abbae09e78016b2c21
I20260809 01:45:49.739749 11063 leader_election.cc:304] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:49.740432 12599 leader_election.cc:290] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:49.740453 12527 raft_consensus.cc:2749] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:49.741042 10581 consensus_peers.cc:597] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 -> Peer 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063): Couldn't send request to peer 71bc72deb1d2402cabd817dfe3b27e88. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260809 01:45:49.742175 11551 tablet_bootstrap.cc:654] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.752346 12545 leader_election.cc:290] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:49.752914 11553 tablet_bootstrap.cc:654] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.753484 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d58189de244802b299e3b376319c1c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:49.753937 10586 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a1e8ce18ca6246b3b89ef824b75cefab has no consensus state
W20260809 01:45:49.754798 10804 leader_election.cc:343] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:49.756645 10596 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c373ab7f921e46a2842ede6258df83af has no consensus state
I20260809 01:45:49.758621 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d58189de244802b299e3b376319c1c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:49.760015 10805 leader_election.cc:343] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.760385 10805 leader_election.cc:304] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:49.761457 12545 raft_consensus.cc:2749] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.762398 11551 tablet_bootstrap.cc:492] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:49.762790 11551 ts_tablet_manager.cc:1403] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.049s user 0.014s sys 0.003s
I20260809 01:45:49.766491 11551 raft_consensus.cc:359] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.767110 11551 raft_consensus.cc:385] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.767419 11551 raft_consensus.cc:740] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.768611 11551 consensus_queue.cc:260] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.779846 11554 tablet_bootstrap.cc:492] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:49.780227 11554 ts_tablet_manager.cc:1403] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.188s user 0.017s sys 0.005s
I20260809 01:45:49.781858 11551 ts_tablet_manager.cc:1434] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.019s user 0.007s sys 0.001s
I20260809 01:45:49.782790 11551 tablet_bootstrap.cc:492] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:49.790525 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 231) took 3962 ms. Trace:
I20260809 01:45:49.790884 11949 rpcz_store.cc:276] 0809 01:45:45.827795 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.828714 (+ 919us) service_pool.cc:224] Handling call
0809 01:45:49.500594 (+3671880us) ts_tablet_manager.cc:1151] Deleting tablet 8323a7a40a1445f980c0adcf6a6d16c7
0809 01:45:49.500639 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:49.622781 (+122142us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.680943 (+ 58162us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.790458 (+109515us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3670930}
I20260809 01:45:49.791877 11554 raft_consensus.cc:359] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.792284 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 8323a7a40a1445f980c0adcf6a6d16c7 (table default.Table_78c91fd32f104ad785b008e345a24e04 [id=dc279b3b97e3489e803a3408d5d76395]) successfully deleted
I20260809 01:45:49.792465 11554 raft_consensus.cc:385] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.792688 11554 raft_consensus.cc:740] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.793249 11554 consensus_queue.cc:260] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.793499 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_197551df65a34e48be825222e02fc1cc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:49.795111 11554 ts_tablet_manager.cc:1434] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.014s user 0.004s sys 0.000s
W20260809 01:45:49.795069 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_197551df65a34e48be825222e02fc1cc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:49.798946 11951 ts_tablet_manager.cc:1929] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:49.799306 11951 log.cc:1199] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/a9d1ed215ba644abbae09e78016b2c21
I20260809 01:45:49.799916 11951 ts_tablet_manager.cc:1950] T a9d1ed215ba644abbae09e78016b2c21 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:49.801311 11551 tablet_bootstrap.cc:654] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.810173 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 232) took 3965 ms. Trace:
I20260809 01:45:49.810477 11949 rpcz_store.cc:276] 0809 01:45:45.844351 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:45.844511 (+ 160us) service_pool.cc:224] Handling call
0809 01:45:49.809771 (+3965260us) ts_tablet_manager.cc:1151] Deleting tablet 66cc1253dee24f5d858d521ddbbfb620
0809 01:45:49.809834 (+ 63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:49.810116 (+ 282us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3964315}
W20260809 01:45:49.811666 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 66cc1253dee24f5d858d521ddbbfb620 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66cc1253dee24f5d858d521ddbbfb620 already in progress: creating tablet
I20260809 01:45:49.813630 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 228) took 3274 ms. Trace:
I20260809 01:45:49.818893 11951 rpcz_store.cc:276] 0809 01:45:46.538799 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.538909 (+ 110us) service_pool.cc:224] Handling call
0809 01:45:49.666439 (+3127530us) ts_tablet_manager.cc:1151] Deleting tablet a9d1ed215ba644abbae09e78016b2c21
0809 01:45:49.666485 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:49.706654 (+ 40169us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.798816 (+ 92162us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.813576 (+ 14760us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3126739}
I20260809 01:45:49.820037 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet a9d1ed215ba644abbae09e78016b2c21 (table default.Table_e111eb8aa0754bdd9e6e571b06d79add [id=78fdf345e656424e9612369430887930]) successfully deleted
I20260809 01:45:49.815742 11554 tablet_bootstrap.cc:492] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:49.821358 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_79eb7d6b90da4bccbad721f2e840dd31"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:49.823477 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_79eb7d6b90da4bccbad721f2e840dd31 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:49.824723 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 229) took 3217 ms. Trace:
I20260809 01:45:49.824999 11951 rpcz_store.cc:276] 0809 01:45:46.607426 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.616258 (+ 8832us) service_pool.cc:224] Handling call
0809 01:45:49.824412 (+3208154us) ts_tablet_manager.cc:1151] Deleting tablet f4d58189de244802b299e3b376319c1c
0809 01:45:49.824453 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:49.824673 (+ 220us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3165602}
W20260809 01:45:49.826087 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet f4d58189de244802b299e3b376319c1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f4d58189de244802b299e3b376319c1c already in progress: creating tablet
I20260809 01:45:49.829442 11554 tablet_bootstrap.cc:654] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.831086 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 230) took 3155 ms. Trace:
I20260809 01:45:49.831496 11951 rpcz_store.cc:276] 0809 01:45:46.675863 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.686025 (+ 10162us) service_pool.cc:224] Handling call
0809 01:45:49.826731 (+3140706us) ts_tablet_manager.cc:1151] Deleting tablet b84f1943c1a34e999fc7c12a5847cdb6
0809 01:45:49.830072 (+ 3341us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:49.831030 (+ 958us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3140108}
I20260809 01:45:49.832348 11553 tablet_bootstrap.cc:492] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:49.832765 11553 ts_tablet_manager.cc:1403] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.174s user 0.007s sys 0.012s
W20260809 01:45:49.834125 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet b84f1943c1a34e999fc7c12a5847cdb6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b84f1943c1a34e999fc7c12a5847cdb6 already in progress: creating tablet
I20260809 01:45:49.835222 11553 raft_consensus.cc:359] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.835867 11553 raft_consensus.cc:385] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.834940 11951 tablet_replica.cc:333] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:49.837248 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_11fb3bbd4a124fcf86837c922880b598"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:49.838294 11951 raft_consensus.cc:2243] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:49.838958 11951 pending_rounds.cc:70] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316: Trying to abort 1 pending ops.
I20260809 01:45:49.839140 11951 pending_rounds.cc:77] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438832890122240 op_type: NO_OP noop_request { }
W20260809 01:45:49.839233 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11fb3bbd4a124fcf86837c922880b598 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:49.839433 11951 raft_consensus.cc:2889] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:49.839710 11951 raft_consensus.cc:2272] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:49.840588 11551 tablet_bootstrap.cc:492] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:49.841032 11551 ts_tablet_manager.cc:1403] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.058s user 0.009s sys 0.007s
I20260809 01:45:49.843372 11551 raft_consensus.cc:359] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.843962 11551 raft_consensus.cc:385] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.844188 11551 raft_consensus.cc:740] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.844764 11551 consensus_queue.cc:260] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.845978 11949 tablet_replica.cc:333] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:49.846881 11949 raft_consensus.cc:2243] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:49.847311 11949 raft_consensus.cc:2272] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:49.850099 11551 ts_tablet_manager.cc:1434] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260809 01:45:49.850926 11551 tablet_bootstrap.cc:492] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:49.851150 11949 ts_tablet_manager.cc:1916] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:49.836113 11553 raft_consensus.cc:740] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.865707 11951 ts_tablet_manager.cc:1916] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:49.868693 12583 raft_consensus.cc:493] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.868955 11551 tablet_bootstrap.cc:654] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.869172 12583 raft_consensus.cc:515] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.871279 12583 leader_election.cc:290] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:49.874234 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cef88ffe9e95437da81793795742def3" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:49.874790 11156 raft_consensus.cc:2468] T cef88ffe9e95437da81793795742def3 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:49.875842 11283 leader_election.cc:304] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:49.876564 12583 raft_consensus.cc:2804] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:49.876842 12583 raft_consensus.cc:493] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:49.877127 12583 raft_consensus.cc:3060] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:49.878227 12545 raft_consensus.cc:493] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.878700 12545 raft_consensus.cc:515] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.880358 11551 tablet_bootstrap.cc:492] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:49.880626 12545 leader_election.cc:290] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:49.880780 11551 ts_tablet_manager.cc:1403] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.009s
I20260809 01:45:49.881892 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0354e4c7bf354f22a9c3c389e0e8ec92" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:49.882030 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cef88ffe9e95437da81793795742def3" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:49.882622 10908 raft_consensus.cc:2468] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:49.882951 12299 raft_consensus.cc:493] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.883112 11551 raft_consensus.cc:359] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.883726 11551 raft_consensus.cc:385] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.883389 12299 raft_consensus.cc:515] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:49.883925 10804 leader_election.cc:343] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.883973 11551 raft_consensus.cc:740] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.884714 11551 consensus_queue.cc:260] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.885998 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0354e4c7bf354f22a9c3c389e0e8ec92" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:49.886684 11551 ts_tablet_manager.cc:1434] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 01:45:49.887529 11551 tablet_bootstrap.cc:492] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
W20260809 01:45:49.888756 10805 leader_election.cc:343] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.888973 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f86bba4bd94fbcacdc9c139e9de2d7" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:49.889108 10805 leader_election.cc:304] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:49.885342 12299 leader_election.cc:290] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:49.891520 11283 leader_election.cc:343] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.893062 11551 tablet_bootstrap.cc:654] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:49.894487 12545 raft_consensus.cc:2749] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:49.895602 11289 leader_election.cc:343] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: c2f86bba4bd94fbcacdc9c139e9de2d7
I20260809 01:45:49.895926 11289 leader_election.cc:304] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:49.896950 12299 raft_consensus.cc:2749] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.869565 11553 consensus_queue.cc:260] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.900692 11553 ts_tablet_manager.cc:1434] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.068s user 0.007s sys 0.000s
I20260809 01:45:49.905081 11553 tablet_bootstrap.cc:492] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:49.912569 12583 raft_consensus.cc:515] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.922945 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_cfd1f608be9041d185710895f202f65e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:49.924465 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cfd1f608be9041d185710895f202f65e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:49.928818 11949 ts_tablet_manager.cc:1929] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:49.929210 11949 log.cc:1199] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/8f0d79e03e66410487a72f1b47b327a0
I20260809 01:45:49.929876 11949 ts_tablet_manager.cc:1950] T 8f0d79e03e66410487a72f1b47b327a0 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:49.934536 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cef88ffe9e95437da81793795742def3" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:49.935218 11156 raft_consensus.cc:3060] T cef88ffe9e95437da81793795742def3 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:49.942077 12545 raft_consensus.cc:493] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.942560 12545 raft_consensus.cc:515] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.944566 12545 leader_election.cc:290] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:49.944464 11156 raft_consensus.cc:2468] T cef88ffe9e95437da81793795742def3 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:49.945961 11283 leader_election.cc:304] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:49.946231 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81da8d08131d4a3ba636ecfc7a7d2951" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:49.947006 12299 raft_consensus.cc:2804] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:49.948493 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81da8d08131d4a3ba636ecfc7a7d2951" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:49.949131 10805 leader_election.cc:343] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:49.950311 10591 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 47) took 860 ms (client timeout 1000 ms). Trace:
W20260809 01:45:49.950615 10591 rpcz_store.cc:269] 0809 01:45:49.089887 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:49.090383 (+ 496us) service_pool.cc:224] Handling call
0809 01:45:49.950227 (+859844us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
W20260809 01:45:49.952539 10804 leader_election.cc:343] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:49.952855 10595 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.66:57751 (request call id 53) took 1020 ms (client timeout 1000 ms). Trace:
I20260809 01:45:49.952999 10804 leader_election.cc:304] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:49.953143 10595 rpcz_store.cc:269] 0809 01:45:48.932649 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:48.932757 (+ 108us) service_pool.cc:224] Handling call
0809 01:45:49.952781 (+1020024us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260809 01:45:49.954962 12545 raft_consensus.cc:2749] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.954708 10592 catalog_manager.cc:5719] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67). New cstate: current_term: 1 leader_uuid: "4f60c9ac9414403d9ea5a97b103c2316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:49.955747 12299 raft_consensus.cc:697] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:49.955889 10592 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 7df6343423604a00a46f203f61e98e2f (table default.Table_18e8960a1b8a457fb622abd881d541ed [id=b7fa92e91f85479c82ba9d384c44d429]). Expected version 1 got 0
I20260809 01:45:49.956830 12299 consensus_queue.cc:237] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.962352 11554 tablet_bootstrap.cc:492] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:49.962982 11554 ts_tablet_manager.cc:1403] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.153s user 0.011s sys 0.000s
I20260809 01:45:49.969091 11118 tablet_service.cc:1511] Processing CreateTablet for tablet 45152843b66046a4936dab73a0fd2027 (DEFAULT_TABLE table=default.Table_e703629aad544e88a395f30f87f6f139 [id=3ab81d1349b04ea28884a4dfd635386f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:49.970098 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45152843b66046a4936dab73a0fd2027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:49.973006 12583 leader_election.cc:290] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:49.975021 11554 raft_consensus.cc:359] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.980275 11554 raft_consensus.cc:385] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:49.980597 11554 raft_consensus.cc:740] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:49.982957 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_52e499c473a24748b65fa3b389cf857a" } new_table_name: "default.Table_15c9a494eb954149a4095ebfca7505a6" modify_external_catalogs: true
I20260809 01:45:49.989711 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 117d5fad26fd42dab19cfc1687364506
I20260809 01:45:49.989773 12545 raft_consensus.cc:493] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:49.990175 12545 raft_consensus.cc:515] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:49.991210 10589 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 323) took 1286 ms. Trace:
I20260809 01:45:49.993083 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63c1a610663a4a2fa58f6cdc5f8d86ce" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:49.994441 10805 leader_election.cc:343] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:49.994899 12545 leader_election.cc:290] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:49.996135 10877 tablet_service.cc:1558] Processing DeleteTablet for tablet 117d5fad26fd42dab19cfc1687364506 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:48 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:49.995560 10591 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_19f955b06eb24b51815571c6ae99966b" } new_table_name: "default.Table_199eacafb2d74a8190d24018b14db258" modify_external_catalogs: true
I20260809 01:45:49.994563 11105 tablet_service.cc:1558] Processing DeleteTablet for tablet 117d5fad26fd42dab19cfc1687364506 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:48 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:49.998584 11347 tablet_service.cc:1558] Processing DeleteTablet for tablet 117d5fad26fd42dab19cfc1687364506 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:48 UTC) from {username='slave'} at 127.0.0.1:53476
W20260809 01:45:49.999043 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 117d5fad26fd42dab19cfc1687364506 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 117d5fad26fd42dab19cfc1687364506 already in progress: creating tablet
I20260809 01:45:50.000031 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63c1a610663a4a2fa58f6cdc5f8d86ce" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:50.001914 10804 leader_election.cc:343] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:50.002326 10804 leader_election.cc:304] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:50.005123 12545 raft_consensus.cc:493] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:50.005105 11105 tablet_service.cc:1511] Processing CreateTablet for tablet 7634fe763a4e4fec91c5938e45a5d1fc (DEFAULT_TABLE table=default.Table_70566a89a3aa4b61884490650a219b50 [id=db841068ce4e47b89f7791b3639f73c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.005553 12545 raft_consensus.cc:515] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.006531 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7634fe763a4e4fec91c5938e45a5d1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.006872 11347 tablet_service.cc:1511] Processing CreateTablet for tablet 45152843b66046a4936dab73a0fd2027 (DEFAULT_TABLE table=default.Table_e703629aad544e88a395f30f87f6f139 [id=3ab81d1349b04ea28884a4dfd635386f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.007978 11347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45152843b66046a4936dab73a0fd2027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.008658 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fffa48ccf58476da79dccd9c008b67c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:50.009955 10805 leader_election.cc:343] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:50.017681 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 14cd02276f26405c95f028edf967bf10
I20260809 01:45:50.011521 12545 leader_election.cc:290] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:50.018924 12545 raft_consensus.cc:2749] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:49.992645 11554 consensus_queue.cc:260] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.025043 11554 ts_tablet_manager.cc:1434] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.062s user 0.006s sys 0.000s
I20260809 01:45:50.026237 11554 tablet_bootstrap.cc:492] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:50.030664 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 14cd02276f26405c95f028edf967bf10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:49 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:50.033855 11124 tablet_service.cc:1558] Processing DeleteTablet for tablet 14cd02276f26405c95f028edf967bf10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:49 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:50.035811 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fffa48ccf58476da79dccd9c008b67c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:50.036449 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet 14cd02276f26405c95f028edf967bf10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:49 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:50.036602 10877 tablet_service.cc:1511] Processing CreateTablet for tablet 45152843b66046a4936dab73a0fd2027 (DEFAULT_TABLE table=default.Table_e703629aad544e88a395f30f87f6f139 [id=3ab81d1349b04ea28884a4dfd635386f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:50.035569 10576 catalog_manager.cc:5017] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 14cd02276f26405c95f028edf967bf10 because the tablet was not found. No further retry: Not found: Tablet not found: 14cd02276f26405c95f028edf967bf10
W20260809 01:45:50.037660 10804 leader_election.cc:343] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:50.040859 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_9e3630f9d80845d5bc57f0dde0550a41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:50.042508 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e3630f9d80845d5bc57f0dde0550a41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:49.991482 10589 rpcz_store.cc:276] 0809 01:45:48.704366 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:48.704474 (+ 108us) service_pool.cc:224] Handling call
0809 01:45:48.706310 (+ 1836us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:48.809896 (+103586us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:48.810013 (+ 117us) catalog_manager.cc:2850] Locking tablets
0809 01:45:49.944525 (+1134512us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:49.989326 (+ 44801us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:49.989512 (+ 186us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:49.989630 (+ 118us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:49.991151 (+ 1521us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:49.963411 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:49.963806 (+ 395us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:49.963828 (+ 22us) write_op.cc:435] Acquired schema lock
0809 01:45:49.963840 (+ 12us) tablet.cc:662] Decoding operations
0809 01:45:49.964146 (+ 306us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:49.964183 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:49.964204 (+ 21us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:49.964469 (+ 265us) tablet.cc:701] Row locks acquired
0809 01:45:49.964482 (+ 13us) write_op.cc:260] PREPARE: finished
0809 01:45:49.964581 (+ 99us) write_op.cc:270] Start()
0809 01:45:49.964655 (+ 74us) write_op.cc:276] Timestamp: P: 1786239949964546 usec, L: 0
0809 01:45:49.964670 (+ 15us) op_driver.cc:348] REPLICATION: starting
0809 01:45:49.965124 (+ 454us) log.cc:844] Serialized 725 byte log entry
0809 01:45:49.986976 (+ 21852us) op_driver.cc:464] REPLICATION: finished
0809 01:45:49.987257 (+ 281us) write_op.cc:301] APPLY: starting
0809 01:45:49.987315 (+ 58us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:49.987419 (+ 104us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:49.987514 (+ 95us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:49.987774 (+ 260us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:49.987925 (+ 151us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:49.987966 (+ 41us) write_op.cc:312] APPLY: finished
0809 01:45:49.988137 (+ 171us) log.cc:844] Serialized 29 byte log entry
0809 01:45:49.988270 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:49.988335 (+ 65us) write_op.cc:454] Released schema lock
0809 01:45:49.988365 (+ 30us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":1188096,"child_traces":[["op",{"apply.queue_time_us":138,"apply.run_cpu_time_us":1289,"apply.run_wall_time_us":1289,"num_ops":2,"prepare.queue_time_us":17117,"prepare.run_cpu_time_us":1966,"prepare.run_wall_time_us":1967,"raft.queue_time_us":12428,"raft.run_cpu_time_us":693,"raft.run_wall_time_us":694,"replication_time_us":22214}]]}
I20260809 01:45:50.038040 10804 leader_election.cc:304] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:50.052551 12545 raft_consensus.cc:493] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:50.053099 12545 raft_consensus.cc:515] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.056835 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b297f9083744e2953d41cd7b1055df" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:50.058597 10805 leader_election.cc:343] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:50.060791 11951 ts_tablet_manager.cc:1929] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:50.061116 11951 log.cc:1199] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/5af35cf31a66474cb0575b2fb1547616
I20260809 01:45:50.061849 11951 ts_tablet_manager.cc:1950] T 5af35cf31a66474cb0575b2fb1547616 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
W20260809 01:45:50.061841 10592 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.67:40463 (request call id 52) took 1004 ms (client timeout 1000 ms). Trace:
I20260809 01:45:50.062641 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 231) took 3356 ms. Trace:
I20260809 01:45:50.063014 11951 rpcz_store.cc:276] 0809 01:45:46.705984 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.706266 (+ 282us) service_pool.cc:224] Handling call
0809 01:45:49.834807 (+3128541us) ts_tablet_manager.cc:1151] Deleting tablet 5af35cf31a66474cb0575b2fb1547616
0809 01:45:49.834845 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:50.019495 (+184650us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:50.060673 (+ 41178us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:50.062572 (+ 1899us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3122539}
I20260809 01:45:50.064158 11951 tablet_replica.cc:333] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:50.064845 11951 raft_consensus.cc:2243] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:50.065729 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 5af35cf31a66474cb0575b2fb1547616 (table default.Table_8031d30fd4be45869a2e535707f3459b [id=ab949c442b89438cbd00a59630ab23e3]) successfully deleted
W20260809 01:45:50.062253 10592 rpcz_store.cc:269] 0809 01:45:49.057695 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:49.057800 (+ 105us) service_pool.cc:224] Handling call
0809 01:45:50.061777 (+1003977us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:49.980679 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:50.013749 (+ 33070us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:50.013774 (+ 25us) write_op.cc:435] Acquired schema lock
0809 01:45:50.013785 (+ 11us) tablet.cc:662] Decoding operations
0809 01:45:50.014043 (+ 258us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:50.014079 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:50.014104 (+ 25us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:50.014235 (+ 131us) tablet.cc:701] Row locks acquired
0809 01:45:50.014250 (+ 15us) write_op.cc:260] PREPARE: finished
0809 01:45:50.014355 (+ 105us) write_op.cc:270] Start()
0809 01:45:50.014435 (+ 80us) write_op.cc:276] Timestamp: P: 1786239950014314 usec, L: 0
0809 01:45:50.014450 (+ 15us) op_driver.cc:348] REPLICATION: starting
0809 01:45:50.014869 (+ 419us) log.cc:844] Serialized 527 byte log entry
0809 01:45:50.052611 (+ 37742us) op_driver.cc:464] REPLICATION: finished
0809 01:45:50.057754 (+ 5143us) write_op.cc:301] APPLY: starting
0809 01:45:50.057862 (+ 108us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:50.057972 (+ 110us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:50.058083 (+ 111us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:50.058266 (+ 183us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:50.058441 (+ 175us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:50.058498 (+ 57us) write_op.cc:312] APPLY: finished
0809 01:45:50.058705 (+ 207us) log.cc:844] Serialized 23 byte log entry
0809 01:45:50.058888 (+ 183us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:50.058956 (+ 68us) write_op.cc:454] Released schema lock
0809 01:45:50.058993 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:50.059904 (+ 911us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:50.059940 (+ 36us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":4948,"apply.run_cpu_time_us":1535,"apply.run_wall_time_us":2130,"num_ops":1,"prepare.queue_time_us":23321,"prepare.run_cpu_time_us":1681,"prepare.run_wall_time_us":34352,"replication_time_us":38066}]]}
I20260809 01:45:50.067658 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 7634fe763a4e4fec91c5938e45a5d1fc (DEFAULT_TABLE table=default.Table_70566a89a3aa4b61884490650a219b50 [id=db841068ce4e47b89f7791b3639f73c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.068593 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7634fe763a4e4fec91c5938e45a5d1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.068482 10595 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 7df6343423604a00a46f203f61e98e2f (table default.Table_18e8960a1b8a457fb622abd881d541ed [id=b7fa92e91f85479c82ba9d384c44d429]). Expected version 1 got 0
I20260809 01:45:50.069629 10595 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet cef88ffe9e95437da81793795742def3 (table default.Table_c54e4ecef7c145cc970af6169c5a9326 [id=40cb4403a8a94bd3938983441e9254d4]). Expected version 1 got 0
I20260809 01:45:50.072256 11554 tablet_bootstrap.cc:654] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:50.074503 11951 raft_consensus.cc:2272] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:50.076370 11951 ts_tablet_manager.cc:1916] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:50.077383 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_65902a8894274e2ca34e86aa908b12f4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:50.080540 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b297f9083744e2953d41cd7b1055df" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:50.082460 12649 raft_consensus.cc:2749] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:50.084537 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_55937d2e689c4822b6d1c366784f19cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:50.086216 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65902a8894274e2ca34e86aa908b12f4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:50.086347 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55937d2e689c4822b6d1c366784f19cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:50.084403 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 7634fe763a4e4fec91c5938e45a5d1fc (DEFAULT_TABLE table=default.Table_70566a89a3aa4b61884490650a219b50 [id=db841068ce4e47b89f7791b3639f73c4]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:50.085284 10804 leader_election.cc:343] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:50.088481 10804 leader_election.cc:304] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:50.085635 12645 consensus_queue.cc:1048] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260809 01:45:50.089680 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cef88ffe9e95437da81793795742def3" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:50.097244 10908 raft_consensus.cc:3060] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:50.101434 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 235) took 3627 ms. Trace:
I20260809 01:45:50.101970 11949 rpcz_store.cc:276] 0809 01:45:46.474180 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.474321 (+ 141us) service_pool.cc:224] Handling call
0809 01:45:49.845802 (+3371481us) ts_tablet_manager.cc:1151] Deleting tablet 8f0d79e03e66410487a72f1b47b327a0
0809 01:45:49.845852 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:49.897824 (+ 51972us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:49.928698 (+ 30874us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:50.101300 (+172602us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3327158}
I20260809 01:45:50.105717 12645 raft_consensus.cc:493] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:50.094192 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7634fe763a4e4fec91c5938e45a5d1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.106155 12645 raft_consensus.cc:515] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.106734 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45152843b66046a4936dab73a0fd2027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.111243 12645 leader_election.cc:290] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:50.105182 10908 raft_consensus.cc:2468] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:50.087893 12649 raft_consensus.cc:493] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:50.059147 12545 leader_election.cc:290] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:50.121564 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 8f0d79e03e66410487a72f1b47b327a0 (table default.Table_30d779db722c451baacdc0202550d5b6 [id=869fc1e71c7142c5bbd1fa358c94e453]) successfully deleted
I20260809 01:45:50.119889 12654 raft_consensus.cc:493] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:50.122088 12545 raft_consensus.cc:2749] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:50.122146 12654 raft_consensus.cc:515] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.119421 12649 raft_consensus.cc:515] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.124121 12654 leader_election.cc:290] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:50.124439 12649 leader_election.cc:290] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:50.125507 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6c244108a4ff3a2df0585e5280294" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:50.126255 11951 ts_tablet_manager.cc:1929] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:50.126590 11951 log.cc:1199] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/135d3d65ea7846d3bc2cb59d44dbedf6
I20260809 01:45:50.127311 11951 ts_tablet_manager.cc:1950] T 135d3d65ea7846d3bc2cb59d44dbedf6 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:50.127404 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a5ef5a3ff44f74ab19998ba9f19310" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:50.128599 10804 leader_election.cc:343] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:50.129179 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 232) took 3171 ms. Trace:
W20260809 01:45:50.129159 10804 leader_election.cc:343] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:50.075311 11105 tablet_service.cc:1511] Processing CreateTablet for tablet a1e8ce18ca6246b3b89ef824b75cefab (DEFAULT_TABLE table=default.Table_6bc9094f4ac34c81b10b00bca15191f1 [id=3bd46c9b1a594b2b9a67e9b4dcfda570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.131417 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1e8ce18ca6246b3b89ef824b75cefab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.131759 11346 tablet_service.cc:1511] Processing CreateTablet for tablet a1e8ce18ca6246b3b89ef824b75cefab (DEFAULT_TABLE table=default.Table_6bc9094f4ac34c81b10b00bca15191f1 [id=3bd46c9b1a594b2b9a67e9b4dcfda570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.135645 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1e8ce18ca6246b3b89ef824b75cefab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.136233 11337 tablet_service.cc:1511] Processing CreateTablet for tablet 9767974e17044c7cb88500ecf5ec621f (DEFAULT_TABLE table=default.Table_08fb89aaa99c421b9236ced8b7b17362 [id=db202b5c3a194763a8d93cdc62d19059]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.137252 11337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9767974e17044c7cb88500ecf5ec621f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.142320 10864 tablet_service.cc:1511] Processing CreateTablet for tablet a1e8ce18ca6246b3b89ef824b75cefab (DEFAULT_TABLE table=default.Table_6bc9094f4ac34c81b10b00bca15191f1 [id=3bd46c9b1a594b2b9a67e9b4dcfda570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.143391 10864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1e8ce18ca6246b3b89ef824b75cefab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.143642 12599 consensus_queue.cc:1048] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:50.134702 11551 tablet_bootstrap.cc:492] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:50.145831 11551 ts_tablet_manager.cc:1403] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.258s user 0.082s sys 0.083s
I20260809 01:45:50.134893 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 9767974e17044c7cb88500ecf5ec621f (DEFAULT_TABLE table=default.Table_08fb89aaa99c421b9236ced8b7b17362 [id=db202b5c3a194763a8d93cdc62d19059]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.147931 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9767974e17044c7cb88500ecf5ec621f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.156695 11949 tablet_replica.cc:333] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:50.157672 11949 raft_consensus.cc:2243] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:50.158126 11949 raft_consensus.cc:2272] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:50.159756 11949 ts_tablet_manager.cc:1916] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:50.170810 10858 tablet_service.cc:1511] Processing CreateTablet for tablet 9767974e17044c7cb88500ecf5ec621f (DEFAULT_TABLE table=default.Table_08fb89aaa99c421b9236ced8b7b17362 [id=db202b5c3a194763a8d93cdc62d19059]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.171903 11551 raft_consensus.cc:359] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.172632 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9767974e17044c7cb88500ecf5ec621f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.172631 11551 raft_consensus.cc:385] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:50.172971 11551 raft_consensus.cc:740] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:50.176270 12650 raft_consensus.cc:493] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:50.176977 12650 raft_consensus.cc:515] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.185287 11551 consensus_queue.cc:260] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.191568 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84f1943c1a34e999fc7c12a5847cdb6" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:50.193172 10804 leader_election.cc:343] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:50.201251 11951 rpcz_store.cc:276] 0809 01:45:46.957861 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.958001 (+ 140us) service_pool.cc:224] Handling call
0809 01:45:50.064031 (+3106030us) ts_tablet_manager.cc:1151] Deleting tablet 135d3d65ea7846d3bc2cb59d44dbedf6
0809 01:45:50.064068 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:50.116349 (+ 52281us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:50.126148 (+ 9799us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:50.129124 (+ 2976us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3096191}
I20260809 01:45:50.203375 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 135d3d65ea7846d3bc2cb59d44dbedf6 (table default.Table_53624396796f4684acbc3787e6e05f6e [id=18e87d9e1fa3442aaa489e3abcd4640e]) successfully deleted
I20260809 01:45:50.205089 11551 ts_tablet_manager.cc:1434] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.059s user 0.006s sys 0.004s
I20260809 01:45:50.205039 11110 tablet_service.cc:1511] Processing CreateTablet for tablet 6b94950efa2a49588fc2a69082e33acb (DEFAULT_TABLE table=default.Table_7f67458772ff440387609b94eba60aa4 [id=f9fcc65838374ac6a72791091c35d450]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.206185 11551 tablet_bootstrap.cc:492] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:50.207063 11110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b94950efa2a49588fc2a69082e33acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.211571 11951 tablet_replica.cc:333] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:50.212484 11951 raft_consensus.cc:2243] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:50.219169 11553 tablet_bootstrap.cc:654] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:50.217336 11551 tablet_bootstrap.cc:654] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:50.234763 11115 tablet_service.cc:1558] Processing DeleteTablet for tablet 4754f2e2a35f4a6a91fa333e2cf564e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:50.235652 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b51f999fab24b1ca7451ff543a124a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:48 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:50.236704 12222 tablet_replica.cc:333] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:50.237578 12222 raft_consensus.cc:2243] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:50.238056 10588 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet a38bd344108649ed8e1687f82154000b
I20260809 01:45:50.238178 12222 pending_rounds.cc:70] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7: Trying to abort 1 pending ops.
I20260809 01:45:50.238422 12222 pending_rounds.cc:77] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438832585662464 op_type: NO_OP noop_request { }
I20260809 01:45:50.238831 12222 raft_consensus.cc:2889] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:50.239199 12222 raft_consensus.cc:2272] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:50.239416 10588 catalog_manager.cc:6807] ReplaceTablet: tablet a38bd344108649ed8e1687f82154000b deleted and replaced by tablet 878b7a9d92df4824902b2b15295f65df
I20260809 01:45:50.242187 12222 ts_tablet_manager.cc:1916] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:50.244777 10598 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet ebc209c696ac4c669961f5ed8392af40
I20260809 01:45:50.246225 10598 catalog_manager.cc:6807] ReplaceTablet: tablet ebc209c696ac4c669961f5ed8392af40 deleted and replaced by tablet 74d6b03590c8472db8977bee5962aa6c
I20260809 01:45:50.246677 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet 5050a590fe0240ac8c9f9191ec2475a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:50.248100 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet dc230f25baf94cbf80aa8e2771ec23ba with delete_type TABLET_DATA_DELETED (replaced by tablet 19fe81e277cd4c798a07ceafa0b60566) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:50.249595 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet a9d1ed215ba644abbae09e78016b2c21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:33 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:50.251226 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet 5af35cf31a66474cb0575b2fb1547616 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:50.252660 10598 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_80fa389cbb634927981d6547e77d96fa" } modify_external_catalogs: true
I20260809 01:45:50.253273 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d35f291c92a45899eed0f2f10a3a1d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:50.253311 10598 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_80fa389cbb634927981d6547e77d96fa" } modify_external_catalogs: true
I20260809 01:45:50.254374 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet c16897857df24c6da5744a99d5f687c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:41 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:50.255553 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet a38bd344108649ed8e1687f82154000b with delete_type TABLET_DATA_DELETED (replaced by tablet 878b7a9d92df4824902b2b15295f65df) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:50.256717 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet ebc209c696ac4c669961f5ed8392af40 with delete_type TABLET_DATA_DELETED (replaced by tablet 74d6b03590c8472db8977bee5962aa6c) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:50.259490 11551 tablet_bootstrap.cc:492] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:50.259459 10599 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 7df6343423604a00a46f203f61e98e2f (table default.Table_18e8960a1b8a457fb622abd881d541ed [id=b7fa92e91f85479c82ba9d384c44d429]). Expected version 1 got 0
I20260809 01:45:50.259876 11551 ts_tablet_manager.cc:1403] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.054s user 0.008s sys 0.022s
I20260809 01:45:50.262305 11551 raft_consensus.cc:359] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:50.262809 11124 tablet_service.cc:1558] Processing DeleteTablet for tablet c2f86bba4bd94fbcacdc9c139e9de2d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:50.262912 11551 raft_consensus.cc:385] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:50.263195 11551 raft_consensus.cc:740] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:50.264026 10596 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_4657bf5b97b244a19b9e1f6c242a6552" } modify_external_catalogs: true
I20260809 01:45:50.263965 11551 consensus_queue.cc:260] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:50.264530 10596 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_4657bf5b97b244a19b9e1f6c242a6552" } modify_external_catalogs: true
I20260809 01:45:50.265480 12222 ts_tablet_manager.cc:1929] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:50.266360 12222 log.cc:1199] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/4754f2e2a35f4a6a91fa333e2cf564e4
I20260809 01:45:50.267047 12222 ts_tablet_manager.cc:1950] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:50.265975 11551 ts_tablet_manager.cc:1434] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 01:45:50.270306 12222 tablet_replica.cc:333] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:50.270980 12222 raft_consensus.cc:2243] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:50.271370 12222 raft_consensus.cc:2272] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:50.272567 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6c244108a4ff3a2df0585e5280294" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:50.273231 12222 ts_tablet_manager.cc:1916] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:50.274061 10805 leader_election.cc:343] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:50.274092 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 4754f2e2a35f4a6a91fa333e2cf564e4 (table default.Table_d1d088df5e7d452ab5a8771a8d3679dd [id=bd9dd208af1e4bd596ae7e3a4b219faf]) successfully deleted
I20260809 01:45:50.274503 10805 leader_election.cc:304] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:50.277884 11951 raft_consensus.cc:2272] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:50.277822 12650 leader_election.cc:290] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:50.278522 12649 raft_consensus.cc:2749] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:50.279703 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a5ef5a3ff44f74ab19998ba9f19310" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:50.280349 11551 tablet_bootstrap.cc:492] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:50.281064 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84f1943c1a34e999fc7c12a5847cdb6" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:50.285461 11354 tablet_service.cc:1558] Processing DeleteTablet for tablet a38bd344108649ed8e1687f82154000b with delete_type TABLET_DATA_DELETED (replaced by tablet 878b7a9d92df4824902b2b15295f65df) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:50.286237 11551 tablet_bootstrap.cc:654] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:50.286187 11334 tablet_service.cc:1558] Processing DeleteTablet for tablet ebc209c696ac4c669961f5ed8392af40 with delete_type TABLET_DATA_DELETED (replaced by tablet 74d6b03590c8472db8977bee5962aa6c) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:50.279388 10856 tablet_service.cc:1558] Processing DeleteTablet for tablet ebc209c696ac4c669961f5ed8392af40 with delete_type TABLET_DATA_DELETED (replaced by tablet 74d6b03590c8472db8977bee5962aa6c) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:50.286916 10864 tablet_service.cc:1558] Processing DeleteTablet for tablet a38bd344108649ed8e1687f82154000b with delete_type TABLET_DATA_DELETED (replaced by tablet 878b7a9d92df4824902b2b15295f65df) from {username='slave'} at 127.0.0.1:45982
W20260809 01:45:50.288708 10805 leader_election.cc:343] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:50.289085 10805 leader_election.cc:304] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:50.290148 10805 leader_election.cc:343] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:50.290485 10805 leader_election.cc:304] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:50.291447 12599 raft_consensus.cc:493] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:50.292105 11951 ts_tablet_manager.cc:1916] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:50.291868 12599 raft_consensus.cc:515] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.292661 12649 raft_consensus.cc:2749] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:50.293291 12649 raft_consensus.cc:2749] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:50.297932 12656 log.cc:927] Time spent T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316: Append to log took a long time: real 0.135s user 0.001s sys 0.000s
I20260809 01:45:50.309742 12599 leader_election.cc:290] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:50.310707 11063 leader_election.cc:343] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 5050a590fe0240ac8c9f9191ec2475a1
I20260809 01:45:50.312876 10599 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]). Expected version 3 got 0
I20260809 01:45:50.315222 12222 ts_tablet_manager.cc:1929] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:50.315560 12222 log.cc:1199] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/9b51f999fab24b1ca7451ff543a124a4
I20260809 01:45:50.316147 12222 ts_tablet_manager.cc:1950] T 9b51f999fab24b1ca7451ff543a124a4 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:50.317373 12599 raft_consensus.cc:493] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:50.317813 12599 raft_consensus.cc:515] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.320675 12599 leader_election.cc:290] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:50.322810 11063 leader_election.cc:343] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 6d35f291c92a45899eed0f2f10a3a1d8
I20260809 01:45:50.323050 12222 tablet_replica.cc:333] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:50.323879 12222 raft_consensus.cc:2243] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:50.324340 12222 raft_consensus.cc:2272] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:50.325992 11553 tablet_bootstrap.cc:492] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:50.326380 11553 ts_tablet_manager.cc:1403] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.421s user 0.016s sys 0.006s
I20260809 01:45:50.329443 11551 tablet_bootstrap.cc:492] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:50.329198 11553 raft_consensus.cc:359] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.329881 11551 ts_tablet_manager.cc:1403] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.050s user 0.009s sys 0.004s
I20260809 01:45:50.332106 11551 raft_consensus.cc:359] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.332652 11551 raft_consensus.cc:385] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:50.332880 11551 raft_consensus.cc:740] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:50.338250 11551 consensus_queue.cc:260] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.338868 12222 ts_tablet_manager.cc:1916] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:50.322355 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 9b51f999fab24b1ca7451ff543a124a4 (table default.Table_5264c640dfc04d9bae010805cdaf4ea6 [id=c7b50f12c7934944aceb691a5d7b0609]) successfully deleted
I20260809 01:45:50.347267 11551 ts_tablet_manager.cc:1434] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.017s user 0.003s sys 0.001s
I20260809 01:45:50.351796 11551 tablet_bootstrap.cc:492] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
W20260809 01:45:50.352360 10586 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e5af40994c204c4f82e36a53e00f2dfe consensus state has no leader
W20260809 01:45:50.357542 10591 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d8ca63cedd3b4626949a0ec07bc26290 has no consensus state
I20260809 01:45:50.329923 11553 raft_consensus.cc:385] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:50.359238 11553 raft_consensus.cc:740] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:50.367987 11551 tablet_bootstrap.cc:654] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:50.370186 11118 tablet_service.cc:1511] Processing CreateTablet for tablet 991e8580b8624f34881d56c6cc60600f (DEFAULT_TABLE table=default.Table_11fa48659a604ed6826b8f83c7580660 [id=7e641a166d7b4425b0c6a9e4cafab331]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.371157 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 991e8580b8624f34881d56c6cc60600f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.381808 11551 tablet_bootstrap.cc:492] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:50.382400 11551 ts_tablet_manager.cc:1403] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.031s user 0.012s sys 0.008s
I20260809 01:45:50.390820 11551 raft_consensus.cc:359] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:50.391378 11551 raft_consensus.cc:385] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:50.391630 11551 raft_consensus.cc:740] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:50.392436 11551 consensus_queue.cc:260] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:50.392966 11334 tablet_service.cc:1511] Processing CreateTablet for tablet 6b94950efa2a49588fc2a69082e33acb (DEFAULT_TABLE table=default.Table_7f67458772ff440387609b94eba60aa4 [id=f9fcc65838374ac6a72791091c35d450]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.394131 12299 consensus_queue.cc:1048] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:50.394989 11334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b94950efa2a49588fc2a69082e33acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.405184 11551 ts_tablet_manager.cc:1434] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
W20260809 01:45:50.410645 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 309106us
I20260809 01:45:50.411005 11551 tablet_bootstrap.cc:492] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:50.411968 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d35f291c92a45899eed0f2f10a3a1d8" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:50.412508 11156 raft_consensus.cc:2468] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:50.419647 12222 ts_tablet_manager.cc:1929] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:50.420089 12222 log.cc:1199] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/5050a590fe0240ac8c9f9191ec2475a1
I20260809 01:45:50.420835 12222 ts_tablet_manager.cc:1950] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:50.422636 11283 leader_election.cc:304] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:50.424161 12653 raft_consensus.cc:2804] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:50.424837 12653 raft_consensus.cc:493] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:50.425173 12653 raft_consensus.cc:3060] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:50.425664 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 261378us
I20260809 01:45:50.427268 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5050a590fe0240ac8c9f9191ec2475a1" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:50.428175 11392 raft_consensus.cc:2468] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:50.433372 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d35f291c92a45899eed0f2f10a3a1d8" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:50.434074 11289 leader_election.cc:343] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 6d35f291c92a45899eed0f2f10a3a1d8
I20260809 01:45:50.434324 11949 ts_tablet_manager.cc:1929] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:50.434815 11949 log.cc:1199] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/f4d58189de244802b299e3b376319c1c
I20260809 01:45:50.435636 11949 ts_tablet_manager.cc:1950] T f4d58189de244802b299e3b376319c1c P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:50.438819 12299 consensus_queue.cc:1048] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:50.446964 11062 leader_election.cc:304] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:50.447794 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_a5ce279641de467da9670256492f78e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:50.448140 11062 raft_consensus.cc:2686] T 5050a590fe0240ac8c9f9191ec2475a1 P 33b5652538e64ed58a09993d3188f8d7: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 01:45:50.449258 11551 tablet_bootstrap.cc:654] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:50.449206 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a5ce279641de467da9670256492f78e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:50.452342 12599 raft_consensus.cc:493] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:50.453096 12599 raft_consensus.cc:515] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.455726 11554 tablet_bootstrap.cc:492] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:50.461736 12599 leader_election.cc:290] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:50.470645 12650 raft_consensus.cc:493] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:50.471072 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 236) took 3870 ms. Trace:
I20260809 01:45:50.471181 12650 raft_consensus.cc:515] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:50.471520 11949 rpcz_store.cc:276] 0809 01:45:46.600674 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.601463 (+ 789us) service_pool.cc:224] Handling call
0809 01:45:50.156482 (+3555019us) ts_tablet_manager.cc:1151] Deleting tablet f4d58189de244802b299e3b376319c1c
0809 01:45:50.156567 (+ 85us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:50.319851 (+163284us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:50.434175 (+114324us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:50.470986 (+ 36811us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32919,"tablet-delete.queue_time_us":3501448}
I20260809 01:45:50.473141 12649 raft_consensus.cc:493] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:50.473584 12649 raft_consensus.cc:515] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.474372 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 6b94950efa2a49588fc2a69082e33acb (DEFAULT_TABLE table=default.Table_7f67458772ff440387609b94eba60aa4 [id=f9fcc65838374ac6a72791091c35d450]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.475328 12649 leader_election.cc:290] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:50.475488 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b94950efa2a49588fc2a69082e33acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.476155 11949 tablet_replica.cc:333] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:50.476840 11949 raft_consensus.cc:2243] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:50.477285 11949 raft_consensus.cc:2272] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:50.479202 11949 ts_tablet_manager.cc:1916] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:50.483055 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet f4d58189de244802b299e3b376319c1c (table default.Table_cf4d3615b483433d81b9cb61a945a25e [id=37ec9a8b5ec74d7fbd8bd053d283a573]) successfully deleted
I20260809 01:45:50.483280 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 5050a590fe0240ac8c9f9191ec2475a1 (table default.Table_fe1acd943c5c42c9b09a0a5845cbee68 [id=487f10bf79a944168eece2b1e8a2daf0]) successfully deleted
I20260809 01:45:50.484222 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7706838afc9f4dbebc3213b82c4da126" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:50.484346 11283 consensus_peers.cc:597] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b51f999fab24b1ca7451ff543a124a4. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:50.484328 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7a8b6d06064addb02979a7feedf756" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:50.485052 12222 tablet_replica.cc:333] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:50.484889 11155 raft_consensus.cc:2468] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 0.
I20260809 01:45:50.473268 12650 leader_election.cc:290] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:50.485857 12222 raft_consensus.cc:2243] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:50.486531 12222 pending_rounds.cc:70] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7: Trying to abort 1 pending ops.
I20260809 01:45:50.486714 12222 pending_rounds.cc:77] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438833904209920 op_type: NO_OP noop_request { }
I20260809 01:45:50.486989 12222 raft_consensus.cc:2889] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:50.487269 12222 raft_consensus.cc:2272] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:50.496418 12222 ts_tablet_manager.cc:1916] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:50.456192 11554 ts_tablet_manager.cc:1403] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.430s user 0.009s sys 0.006s
I20260809 01:45:50.500922 11551 tablet_bootstrap.cc:492] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:50.501314 11551 ts_tablet_manager.cc:1403] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.090s user 0.012s sys 0.004s
I20260809 01:45:50.500967 11553 consensus_queue.cc:260] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:50.503350 10804 leader_election.cc:343] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:50.503975 10804 leader_election.cc:304] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:50.504346 11551 raft_consensus.cc:359] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.504946 11551 raft_consensus.cc:385] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:50.505241 11551 raft_consensus.cc:740] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:50.506029 11551 consensus_queue.cc:260] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.507017 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6159d660d544b3e907a08372027c0e2" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:50.509060 11384 raft_consensus.cc:2468] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:50.509369 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7a8b6d06064addb02979a7feedf756" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:50.510638 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7706838afc9f4dbebc3213b82c4da126" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:50.511361 10805 leader_election.cc:343] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:50.511783 10805 leader_election.cc:304] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:50.514500 11553 ts_tablet_manager.cc:1434] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.188s user 0.007s sys 0.001s
I20260809 01:45:50.514314 11062 leader_election.cc:304] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:50.515478 12599 raft_consensus.cc:2804] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:50.515959 12599 raft_consensus.cc:493] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:50.516450 12599 raft_consensus.cc:3060] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:50.517730 11553 tablet_bootstrap.cc:492] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:50.515677 12545 raft_consensus.cc:2749] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:50.519333 11105 tablet_service.cc:1511] Processing CreateTablet for tablet 42d00caf5a784b19b4b09914d46d2337 (DEFAULT_TABLE table=default.Table_25bc6c83b5ad43ef8b4fb22d70970c6a [id=f52d538e384f41a6abb5f3bd64c78cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.521127 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42d00caf5a784b19b4b09914d46d2337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.520648 10858 tablet_service.cc:1511] Processing CreateTablet for tablet 991e8580b8624f34881d56c6cc60600f (DEFAULT_TABLE table=default.Table_11fa48659a604ed6826b8f83c7580660 [id=7e641a166d7b4425b0c6a9e4cafab331]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.522394 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 991e8580b8624f34881d56c6cc60600f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.515849 12654 raft_consensus.cc:2804] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:50.527719 12654 raft_consensus.cc:493] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:50.528283 12654 raft_consensus.cc:3060] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:50.530834 11551 ts_tablet_manager.cc:1434] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.029s user 0.005s sys 0.000s
I20260809 01:45:50.543038 12599 raft_consensus.cc:515] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.545840 11551 tablet_bootstrap.cc:492] T 83ac8498dff04b4e93303d7a35e62134 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
W20260809 01:45:50.553720 10805 leader_election.cc:343] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:50.555377 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6159d660d544b3e907a08372027c0e2" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:50.556216 11395 raft_consensus.cc:3060] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:50.567975 12599 leader_election.cc:290] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:50.549614 12653 raft_consensus.cc:515] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:50.573477 10581 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet d8ca63cedd3b4626949a0ec07bc26290 (table default.Table_199eacafb2d74a8190d24018b14db258 [id=e224e231c3e64f1ca6887a3593acf948], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d8ca63cedd3b4626949a0ec07bc26290 has no consensus state
I20260809 01:45:50.573809 11392 raft_consensus.cc:2393] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33b5652538e64ed58a09993d3188f8d7 in current term 1: Already voted for candidate 4f60c9ac9414403d9ea5a97b103c2316 in this term.
I20260809 01:45:50.573894 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d35f291c92a45899eed0f2f10a3a1d8" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:50.574746 11155 raft_consensus.cc:3060] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:50.578585 11062 leader_election.cc:304] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:50.580207 11289 leader_election.cc:343] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 6d35f291c92a45899eed0f2f10a3a1d8
I20260809 01:45:50.586162 12653 leader_election.cc:290] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:50.591699 10599 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]). Expected version 3 got 0
I20260809 01:45:50.591821 11155 raft_consensus.cc:2468] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:50.592512 12599 raft_consensus.cc:2749] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:50.598308 11553 tablet_bootstrap.cc:654] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:50.598681 11551 tablet_bootstrap.cc:654] T 83ac8498dff04b4e93303d7a35e62134 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:50.610247 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_1d10d5edc68f414f89a9b12b052c7aae" } new_table_name: "default.Table_8060ad8eb1be43f4a3c2e9dac705c0e3" modify_external_catalogs: true
I20260809 01:45:50.614923 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_f9ca1c8245e643c994d68dad6b152b09" } new_table_name: "default.Table_14f169b63955470587dbbdaea713291d" modify_external_catalogs: true
I20260809 01:45:50.623369 10595 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 7df6343423604a00a46f203f61e98e2f (table default.Table_18e8960a1b8a457fb622abd881d541ed [id=b7fa92e91f85479c82ba9d384c44d429]). Expected version 1 got 0
I20260809 01:45:50.624161 10595 catalog_manager.cc:5719] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67). New cstate: current_term: 1 leader_uuid: "4f60c9ac9414403d9ea5a97b103c2316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:50.625540 10595 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet cef88ffe9e95437da81793795742def3 (table default.Table_c54e4ecef7c145cc970af6169c5a9326 [id=40cb4403a8a94bd3938983441e9254d4]). Expected version 1 got 0
I20260809 01:45:50.625686 12654 raft_consensus.cc:515] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:50.628886 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7706838afc9f4dbebc3213b82c4da126" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:50.629586 11158 raft_consensus.cc:3060] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:50.638618 11949 ts_tablet_manager.cc:1929] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:50.638999 11949 log.cc:1199] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/b84f1943c1a34e999fc7c12a5847cdb6
I20260809 01:45:50.639631 11949 ts_tablet_manager.cc:1950] T b84f1943c1a34e999fc7c12a5847cdb6 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:50.640357 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 237) took 3960 ms. Trace:
I20260809 01:45:50.640635 11949 rpcz_store.cc:276] 0809 01:45:46.679747 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.688917 (+ 9170us) service_pool.cc:224] Handling call
0809 01:45:50.476011 (+3787094us) ts_tablet_manager.cc:1151] Deleting tablet b84f1943c1a34e999fc7c12a5847cdb6
0809 01:45:50.476065 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:50.552665 (+ 76600us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:50.638511 (+ 85846us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:50.640309 (+ 1798us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3785917}
I20260809 01:45:50.641902 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet b84f1943c1a34e999fc7c12a5847cdb6 (table default.Table_e39314f303ee429f897d4f85b8edbf69 [id=21bc1ee96b724341904d6b0c5b10fb7a]) successfully deleted
I20260809 01:45:50.643103 10877 tablet_service.cc:1511] Processing CreateTablet for tablet 42d00caf5a784b19b4b09914d46d2337 (DEFAULT_TABLE table=default.Table_25bc6c83b5ad43ef8b4fb22d70970c6a [id=f52d538e384f41a6abb5f3bd64c78cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.644116 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42d00caf5a784b19b4b09914d46d2337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.645833 11158 raft_consensus.cc:2468] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 1.
I20260809 01:45:50.650791 10804 leader_election.cc:304] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:50.651523 12649 raft_consensus.cc:2804] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:50.651964 12649 raft_consensus.cc:697] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Becoming Leader. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Running, Role: LEADER
I20260809 01:45:50.652758 12649 consensus_queue.cc:237] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:50.661571 12222 ts_tablet_manager.cc:1929] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:50.662003 12222 log.cc:1199] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/dc230f25baf94cbf80aa8e2771ec23ba
I20260809 01:45:50.662832 12222 ts_tablet_manager.cc:1950] T dc230f25baf94cbf80aa8e2771ec23ba P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:50.665283 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet dc230f25baf94cbf80aa8e2771ec23ba (table default.Table_1998ec81597a4c4584dc4e4f90caa274 [id=cb347b72c1cf436095c6043c5b169641]) successfully deleted
I20260809 01:45:50.665472 12222 tablet_replica.cc:333] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:50.666177 12222 raft_consensus.cc:2243] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:50.666458 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6159d660d544b3e907a08372027c0e2" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:50.667065 10905 raft_consensus.cc:2468] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:50.670904 11124 tablet_service.cc:1511] Processing CreateTablet for tablet fe4afb4101ec445e94210e3722725b59 (DEFAULT_TABLE table=default.Table_55fdbc6110a64bed9f6caafd0b681303 [id=dbf41cf7473944af8689121ae77512a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.671940 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe4afb4101ec445e94210e3722725b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.675132 10864 tablet_service.cc:1511] Processing CreateTablet for tablet fe4afb4101ec445e94210e3722725b59 (DEFAULT_TABLE table=default.Table_55fdbc6110a64bed9f6caafd0b681303 [id=dbf41cf7473944af8689121ae77512a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.679077 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6159d660d544b3e907a08372027c0e2" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:50.679589 10905 raft_consensus.cc:3060] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:50.683111 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_8dde73a0e7f44af28232bbb7a0d57897"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:50.679528 12654 leader_election.cc:290] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:50.681951 11949 tablet_replica.cc:333] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
W20260809 01:45:50.684697 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8dde73a0e7f44af28232bbb7a0d57897 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:50.690659 11116 tablet_service.cc:1511] Processing CreateTablet for tablet 14cd02276f26405c95f028edf967bf10 (DEFAULT_TABLE table=default.Table_056eba06842f4044b4a9f9922d208c59 [id=81bfe444a2504dd6a36e0faa522e1629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.691179 11949 raft_consensus.cc:2243] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:50.692286 11949 raft_consensus.cc:2272] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:50.692647 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14cd02276f26405c95f028edf967bf10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.698745 11551 tablet_bootstrap.cc:492] T 83ac8498dff04b4e93303d7a35e62134 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:50.698865 10905 raft_consensus.cc:2468] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:50.699297 11551 ts_tablet_manager.cc:1403] T 83ac8498dff04b4e93303d7a35e62134 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.168s user 0.001s sys 0.014s
I20260809 01:45:50.701632 12222 raft_consensus.cc:2272] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:50.703704 12222 ts_tablet_manager.cc:1916] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:50.710683 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 14cd02276f26405c95f028edf967bf10 (DEFAULT_TABLE table=default.Table_056eba06842f4044b4a9f9922d208c59 [id=81bfe444a2504dd6a36e0faa522e1629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.721967 11115 tablet_service.cc:1511] Processing CreateTablet for tablet 241ce7d4549848abab373eb22e9aba97 (DEFAULT_TABLE table=default.Table_aa2f13d6f448442aa146a0b68c883765 [id=42ed621ae3be49f490a340f2da56a060]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.723675 11115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 241ce7d4549848abab373eb22e9aba97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.725593 11551 raft_consensus.cc:359] T 83ac8498dff04b4e93303d7a35e62134 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.726323 11551 raft_consensus.cc:385] T 83ac8498dff04b4e93303d7a35e62134 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:50.726613 11551 raft_consensus.cc:740] T 83ac8498dff04b4e93303d7a35e62134 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:50.711036 11949 ts_tablet_manager.cc:1916] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:50.722419 10856 tablet_service.cc:1511] Processing CreateTablet for tablet 241ce7d4549848abab373eb22e9aba97 (DEFAULT_TABLE table=default.Table_aa2f13d6f448442aa146a0b68c883765 [id=42ed621ae3be49f490a340f2da56a060]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.733785 11551 consensus_queue.cc:260] T 83ac8498dff04b4e93303d7a35e62134 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.746157 11063 leader_election.cc:304] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:50.747056 12599 raft_consensus.cc:2804] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:50.630380 11283 leader_election.cc:304] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:50.758412 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 128096us
I20260809 01:45:50.749930 12599 raft_consensus.cc:697] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
W20260809 01:45:50.758277 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 60458us
I20260809 01:45:50.760636 12599 consensus_queue.cc:237] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.767470 11551 ts_tablet_manager.cc:1434] T 83ac8498dff04b4e93303d7a35e62134 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.068s user 0.009s sys 0.001s
I20260809 01:45:50.768967 11551 tablet_bootstrap.cc:492] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:50.746851 11554 raft_consensus.cc:359] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:50.777665 11554 raft_consensus.cc:385] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:50.777979 11554 raft_consensus.cc:740] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:50.782848 11551 tablet_bootstrap.cc:654] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:50.778434 12583 raft_consensus.cc:2804] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:50.820869 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7706838afc9f4dbebc3213b82c4da126" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:50.822508 12649 raft_consensus.cc:493] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:50.822962 12649 raft_consensus.cc:515] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.823292 12222 ts_tablet_manager.cc:1929] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:50.823644 12222 log.cc:1199] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/a9d1ed215ba644abbae09e78016b2c21
I20260809 01:45:50.824191 12222 ts_tablet_manager.cc:1950] T a9d1ed215ba644abbae09e78016b2c21 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:50.827346 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac8498dff04b4e93303d7a35e62134" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:50.827930 11158 raft_consensus.cc:2468] T 83ac8498dff04b4e93303d7a35e62134 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 0.
I20260809 01:45:50.829303 10804 leader_election.cc:304] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:50.830065 12654 raft_consensus.cc:2804] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:50.830375 12654 raft_consensus.cc:493] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:50.830714 12654 raft_consensus.cc:3060] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:50.831326 12222 tablet_replica.cc:333] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:50.831532 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet a9d1ed215ba644abbae09e78016b2c21 (table default.Table_e111eb8aa0754bdd9e6e571b06d79add [id=78fdf345e656424e9612369430887930]) successfully deleted
I20260809 01:45:50.831995 12222 raft_consensus.cc:2243] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:50.832494 12222 raft_consensus.cc:2272] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:50.834739 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac8498dff04b4e93303d7a35e62134" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:50.836491 10805 leader_election.cc:343] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:50.825204 12649 leader_election.cc:290] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:50.837646 12599 raft_consensus.cc:493] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:50.837997 12599 raft_consensus.cc:515] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.839777 12599 leader_election.cc:290] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:50.848575 12654 raft_consensus.cc:515] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.850175 12222 ts_tablet_manager.cc:1916] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:50.850682 11063 leader_election.cc:343] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: c16897857df24c6da5744a99d5f687c0
I20260809 01:45:50.854851 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_65be7c5fe96148898b9fdb468cee042d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:50.856509 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65be7c5fe96148898b9fdb468cee042d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:50.871734 12222 ts_tablet_manager.cc:1929] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:50.868173 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16897857df24c6da5744a99d5f687c0" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:50.872052 12222 log.cc:1199] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/5af35cf31a66474cb0575b2fb1547616
I20260809 01:45:50.872191 11392 raft_consensus.cc:2468] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:50.873395 11062 leader_election.cc:304] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:50.874243 12599 raft_consensus.cc:2804] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:50.874586 12599 raft_consensus.cc:493] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:50.874890 12599 raft_consensus.cc:3060] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:50.880868 12599 raft_consensus.cc:515] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.882792 11116 tablet_service.cc:1511] Processing CreateTablet for tablet 9a354a7cc2a84123a456125ef6c9828a (DEFAULT_TABLE table=default.Table_8d898b49c47e4d0591f1cc0dc2972d36 [id=e04656baa83648f6899b7d05a30f0308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.883845 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a354a7cc2a84123a456125ef6c9828a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.889643 12222 ts_tablet_manager.cc:1950] T 5af35cf31a66474cb0575b2fb1547616 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:50.781057 11554 consensus_queue.cc:260] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:50.892014 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 5af35cf31a66474cb0575b2fb1547616 (table default.Table_8031d30fd4be45869a2e535707f3459b [id=ab949c442b89438cbd00a59630ab23e3]) successfully deleted
I20260809 01:45:50.892905 12599 leader_election.cc:290] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:50.893760 11383 raft_consensus.cc:3060] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:50.786034 11951 ts_tablet_manager.cc:1929] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260809 01:45:50.895334 11951 log.cc:1199] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/ead75d766d0346f6823074d90ba8054e
I20260809 01:45:50.896051 11951 ts_tablet_manager.cc:1950] T ead75d766d0346f6823074d90ba8054e P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:50.900907 11383 raft_consensus.cc:2468] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 1.
I20260809 01:45:50.893153 12222 tablet_replica.cc:333] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
W20260809 01:45:50.894758 11063 leader_election.cc:343] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: c16897857df24c6da5744a99d5f687c0
I20260809 01:45:50.905849 11395 raft_consensus.cc:2468] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:50.909327 12599 raft_consensus.cc:493] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:50.909899 12599 raft_consensus.cc:515] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.910416 10598 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet d431b3f6b22a40e8aad3ef242e9fdd0d
I20260809 01:45:50.912744 12222 raft_consensus.cc:2243] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:50.913161 12222 raft_consensus.cc:2272] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:50.913286 10596 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 1c73bed7a9774cbc8ac774e6514a981d
I20260809 01:45:50.915297 12222 ts_tablet_manager.cc:1916] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:50.924786 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c73bed7a9774cbc8ac774e6514a981d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:50 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:50.925635 12599 leader_election.cc:290] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:50.928309 11063 leader_election.cc:343] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 36a542c773b7438ead16c5b3af224cf9
I20260809 01:45:50.882347 12697 tablet.cc:1724] T 7df6343423604a00a46f203f61e98e2f P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:50.907191 11553 tablet_bootstrap.cc:492] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:50.934753 11553 ts_tablet_manager.cc:1403] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.417s user 0.017s sys 0.027s
I20260809 01:45:50.935940 12654 leader_election.cc:290] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:50.916774 10858 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c73bed7a9774cbc8ac774e6514a981d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:50 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:50.939312 10858 tablet_service.cc:1511] Processing CreateTablet for tablet 9a354a7cc2a84123a456125ef6c9828a (DEFAULT_TABLE table=default.Table_8d898b49c47e4d0591f1cc0dc2972d36 [id=e04656baa83648f6899b7d05a30f0308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:50.941437 10864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe4afb4101ec445e94210e3722725b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.942235 11553 raft_consensus.cc:359] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.943131 11553 raft_consensus.cc:385] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:50.943436 11553 raft_consensus.cc:740] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:50.944380 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a354a7cc2a84123a456125ef6c9828a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.949245 11553 consensus_queue.cc:260] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:50.951131 11553 ts_tablet_manager.cc:1434] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20260809 01:45:50.952060 11553 tablet_bootstrap.cc:492] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:50.959429 10856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 241ce7d4549848abab373eb22e9aba97. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:50.964295 10586 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f4985b1715f64adaae432d46dee4ea38 has no consensus state
I20260809 01:45:50.967064 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac8498dff04b4e93303d7a35e62134" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:50.967895 11158 raft_consensus.cc:3060] T 83ac8498dff04b4e93303d7a35e62134 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:50.968254 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c373ab7f921e46a2842ede6258df83af has no consensus state
I20260809 01:45:50.969983 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14cd02276f26405c95f028edf967bf10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:50.970198 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_0a995cd327c14182b69fd09bec3096d6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:50.971757 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a995cd327c14182b69fd09bec3096d6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:50.928421 11554 ts_tablet_manager.cc:1434] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.429s user 0.007s sys 0.000s
I20260809 01:45:51.015661 10877 tablet_service.cc:1511] Processing CreateTablet for tablet 21dab4386b434338ad694094a6c036a2 (DEFAULT_TABLE table=default.Table_37259f04b8114af3be8d7a5f8484431d [id=fb06ce98849d49d79d3a930dfa91ace0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.018611 11554 tablet_bootstrap.cc:492] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:51.028173 12222 ts_tablet_manager.cc:1929] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:51.028575 12222 log.cc:1199] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/6d35f291c92a45899eed0f2f10a3a1d8
I20260809 01:45:51.029186 12222 ts_tablet_manager.cc:1950] T 6d35f291c92a45899eed0f2f10a3a1d8 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:51.031173 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 234) took 3825 ms. Trace:
I20260809 01:45:51.031649 11951 rpcz_store.cc:276] 0809 01:45:47.205378 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:47.205521 (+ 143us) service_pool.cc:224] Handling call
0809 01:45:50.211416 (+3005895us) ts_tablet_manager.cc:1151] Deleting tablet ead75d766d0346f6823074d90ba8054e
0809 01:45:50.211463 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:50.311426 (+ 99963us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:50.785659 (+474233us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:51.031103 (+245444us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113288,"tablet-delete.queue_time_us":3005140}
I20260809 01:45:51.034441 11110 tablet_service.cc:1511] Processing CreateTablet for tablet 21dab4386b434338ad694094a6c036a2 (DEFAULT_TABLE table=default.Table_37259f04b8114af3be8d7a5f8484431d [id=fb06ce98849d49d79d3a930dfa91ace0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.035673 11110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21dab4386b434338ad694094a6c036a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.037300 11553 tablet_bootstrap.cc:654] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:51.042025 11158 raft_consensus.cc:2468] T 83ac8498dff04b4e93303d7a35e62134 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 1.
I20260809 01:45:51.056658 12654 raft_consensus.cc:493] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:51.057305 12654 raft_consensus.cc:515] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.066603 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977694496e014ef29f5d7d513e5219c0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:51.067853 12654 leader_election.cc:290] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:51.068109 10804 leader_election.cc:343] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:51.033223 11951 tablet_replica.cc:333] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:51.074721 11951 raft_consensus.cc:2243] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:51.075218 11951 pending_rounds.cc:70] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316: Trying to abort 1 pending ops.
I20260809 01:45:51.075459 11951 pending_rounds.cc:77] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438832585662464 op_type: NO_OP noop_request { }
I20260809 01:45:51.075817 11951 raft_consensus.cc:2889] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:51.076171 11951 raft_consensus.cc:2272] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:51.080783 11346 tablet_service.cc:1511] Processing CreateTablet for tablet fe4afb4101ec445e94210e3722725b59 (DEFAULT_TABLE table=default.Table_55fdbc6110a64bed9f6caafd0b681303 [id=dbf41cf7473944af8689121ae77512a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.086418 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 6d35f291c92a45899eed0f2f10a3a1d8 (table default.Table_14613c38aa0e4b258598254918b193bb [id=fa2e19a3042b41959165ca54e46ca690]) successfully deleted
I20260809 01:45:51.087222 12222 tablet_replica.cc:333] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:51.087298 11551 tablet_bootstrap.cc:492] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:51.087669 11551 ts_tablet_manager.cc:1403] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.319s user 0.066s sys 0.073s
I20260809 01:45:51.090046 12654 raft_consensus.cc:493] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:51.090533 12654 raft_consensus.cc:515] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.091176 11551 raft_consensus.cc:359] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.091542 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21dab4386b434338ad694094a6c036a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.091738 11551 raft_consensus.cc:385] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:51.092053 11551 raft_consensus.cc:740] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:51.092763 11551 consensus_queue.cc:260] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.028744 12583 raft_consensus.cc:697] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:51.093489 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d5a709cd86476594613e788a325094" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:51.095432 10804 leader_election.cc:343] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:51.098141 10804 leader_election.cc:304] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:51.099011 12649 raft_consensus.cc:2804] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:51.099229 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 14cd02276f26405c95f028edf967bf10 (DEFAULT_TABLE table=default.Table_056eba06842f4044b4a9f9922d208c59 [id=81bfe444a2504dd6a36e0faa522e1629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.099679 12649 raft_consensus.cc:697] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Becoming Leader. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Running, Role: LEADER
I20260809 01:45:51.100456 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14cd02276f26405c95f028edf967bf10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.100894 12649 consensus_queue.cc:237] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.105158 11551 ts_tablet_manager.cc:1434] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20260809 01:45:51.112128 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 241ce7d4549848abab373eb22e9aba97 (DEFAULT_TABLE table=default.Table_aa2f13d6f448442aa146a0b68c883765 [id=42ed621ae3be49f490a340f2da56a060]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.113181 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 241ce7d4549848abab373eb22e9aba97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.114154 10597 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_70566a89a3aa4b61884490650a219b50" } modify_external_catalogs: true
I20260809 01:45:51.114701 10597 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_70566a89a3aa4b61884490650a219b50" } modify_external_catalogs: true
I20260809 01:45:51.084787 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac8498dff04b4e93303d7a35e62134" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:51.116724 11551 tablet_bootstrap.cc:492] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:51.117228 11553 tablet_bootstrap.cc:492] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:51.118153 11553 ts_tablet_manager.cc:1403] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.166s user 0.010s sys 0.005s
I20260809 01:45:51.119736 10586 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_11fa48659a604ed6826b8f83c7580660" } modify_external_catalogs: true
I20260809 01:45:51.120292 10586 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_11fa48659a604ed6826b8f83c7580660" } modify_external_catalogs: true
I20260809 01:45:51.122072 11118 tablet_service.cc:1511] Processing CreateTablet for tablet afafd99138c54fc6aa76c0cda0901842 (DEFAULT_TABLE table=default.Table_c79f32e8e08f4d73969d16b8846d500a [id=72c4a61d1ec548e49df850ad78e8377d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.123001 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afafd99138c54fc6aa76c0cda0901842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.124886 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c73bed7a9774cbc8ac774e6514a981d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:50 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:51.130272 12222 raft_consensus.cc:2243] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:51.130918 12222 raft_consensus.cc:2272] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:51.133246 12222 ts_tablet_manager.cc:1916] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:51.136839 11551 tablet_bootstrap.cc:654] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:51.141125 12599 raft_consensus.cc:493] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:51.141881 12599 raft_consensus.cc:515] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.143466 10589 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 7df6343423604a00a46f203f61e98e2f (table default.Table_18e8960a1b8a457fb622abd881d541ed [id=b7fa92e91f85479c82ba9d384c44d429]). Expected version 1 got 0
I20260809 01:45:51.146368 10588 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 7df6343423604a00a46f203f61e98e2f (table default.Table_18e8960a1b8a457fb622abd881d541ed [id=b7fa92e91f85479c82ba9d384c44d429]). Expected version 1 got 0
W20260809 01:45:51.148167 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 257035us
I20260809 01:45:51.152172 11105 tablet_service.cc:1558] Processing DeleteTablet for tablet 60d356fcc48441ca80f2df29b533404f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:39 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:51.153098 12599 leader_election.cc:290] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:51.126668 11334 consensus_meta.cc:287] T 6b94950efa2a49588fc2a69082e33acb P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.588s user 0.010s sys 0.000s
W20260809 01:45:51.154845 11063 leader_election.cc:343] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: c2f86bba4bd94fbcacdc9c139e9de2d7
I20260809 01:45:51.163589 10589 catalog_manager.cc:5719] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66). New cstate: current_term: 1 leader_uuid: "33b5652538e64ed58a09993d3188f8d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:51.166272 11124 tablet_service.cc:1558] Processing DeleteTablet for tablet 36a542c773b7438ead16c5b3af224cf9 with delete_type TABLET_DATA_DELETED (replaced by tablet a38bd344108649ed8e1687f82154000b) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:51.182857 10852 tablet_service.cc:1511] Processing CreateTablet for tablet afafd99138c54fc6aa76c0cda0901842 (DEFAULT_TABLE table=default.Table_c79f32e8e08f4d73969d16b8846d500a [id=72c4a61d1ec548e49df850ad78e8377d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.168385 10588 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_8eee5e53a225495ea337c18604165174" } new_table_name: "default.Table_c1a0ae6f926d45f780cc9f26ee4a1d49" modify_external_catalogs: true
I20260809 01:45:51.182564 11347 tablet_service.cc:1511] Processing CreateTablet for tablet 991e8580b8624f34881d56c6cc60600f (DEFAULT_TABLE table=default.Table_11fa48659a604ed6826b8f83c7580660 [id=7e641a166d7b4425b0c6a9e4cafab331]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.190085 11347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 991e8580b8624f34881d56c6cc60600f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.191095 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977694496e014ef29f5d7d513e5219c0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:51.169595 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_51519b5a61474dba91f80b33789a2849"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:51.171501 10592 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_79500ae2591d480f93bd129b5e4e5804" } new_table_name: "default.Table_ed7186006163408da3c63d5ad3528807" modify_external_catalogs: true
I20260809 01:45:51.192334 11337 tablet_service.cc:1511] Processing CreateTablet for tablet 42d00caf5a784b19b4b09914d46d2337 (DEFAULT_TABLE table=default.Table_25bc6c83b5ad43ef8b4fb22d70970c6a [id=f52d538e384f41a6abb5f3bd64c78cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.193689 11337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42d00caf5a784b19b4b09914d46d2337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.195971 11351 tablet_service.cc:1511] Processing CreateTablet for tablet 9a354a7cc2a84123a456125ef6c9828a (DEFAULT_TABLE table=default.Table_8d898b49c47e4d0591f1cc0dc2972d36 [id=e04656baa83648f6899b7d05a30f0308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.197520 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a354a7cc2a84123a456125ef6c9828a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.202750 11124 tablet_service.cc:1511] Processing CreateTablet for tablet d18d1f8343ed4c33a70fccbecc3012a6 (DEFAULT_TABLE table=default.Table_ae40e284667c40089594f8328c9630d7 [id=a510e3ee842f43d8931fe23f018bb7ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.203768 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d18d1f8343ed4c33a70fccbecc3012a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.205727 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe4afb4101ec445e94210e3722725b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.214898 12583 consensus_queue.cc:237] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.222486 12599 raft_consensus.cc:493] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:51.222956 12599 raft_consensus.cc:515] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:51.224565 10855 tablet_service.cc:1511] Processing CreateTablet for tablet d18d1f8343ed4c33a70fccbecc3012a6 (DEFAULT_TABLE table=default.Table_ae40e284667c40089594f8328c9630d7 [id=a510e3ee842f43d8931fe23f018bb7ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.225226 12599 leader_election.cc:290] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:51.228215 11063 leader_election.cc:343] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 60d356fcc48441ca80f2df29b533404f
W20260809 01:45:51.197258 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51519b5a61474dba91f80b33789a2849 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:51.240700 11551 tablet_bootstrap.cc:492] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:51.241230 11551 ts_tablet_manager.cc:1403] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.125s user 0.020s sys 0.014s
I20260809 01:45:51.250066 11551 raft_consensus.cc:359] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.250800 11551 raft_consensus.cc:385] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:51.251092 11551 raft_consensus.cc:740] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:51.253997 11551 consensus_queue.cc:260] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.258998 11551 ts_tablet_manager.cc:1434] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.017s user 0.009s sys 0.001s
I20260809 01:45:51.268818 12222 ts_tablet_manager.cc:1929] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260809 01:45:51.270359 10805 leader_election.cc:343] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:51.279930 11951 ts_tablet_manager.cc:1916] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:51.280786 12708 log.cc:927] Time spent T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316: Append to log took a long time: real 0.207s user 0.001s sys 0.000s
I20260809 01:45:51.295832 12545 raft_consensus.cc:493] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:51.296381 12545 raft_consensus.cc:515] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.316745 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 5050a590fe0240ac8c9f9191ec2475a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:51.316771 11366 tablet_service.cc:1511] Processing CreateTablet for tablet afafd99138c54fc6aa76c0cda0901842 (DEFAULT_TABLE table=default.Table_c79f32e8e08f4d73969d16b8846d500a [id=72c4a61d1ec548e49df850ad78e8377d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.318011 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet c16897857df24c6da5744a99d5f687c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:41 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:51.318063 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afafd99138c54fc6aa76c0cda0901842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.320201 11334 tablet_service.cc:1511] Processing CreateTablet for tablet 21dab4386b434338ad694094a6c036a2 (DEFAULT_TABLE table=default.Table_37259f04b8114af3be8d7a5f8484431d [id=fb06ce98849d49d79d3a930dfa91ace0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.321391 11334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21dab4386b434338ad694094a6c036a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.343478 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16897857df24c6da5744a99d5f687c0" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:51.344447 11395 raft_consensus.cc:3060] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:51.269866 11551 tablet_bootstrap.cc:492] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:51.348044 12222 log.cc:1199] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/c16897857df24c6da5744a99d5f687c0
I20260809 01:45:51.348892 12222 ts_tablet_manager.cc:1950] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:51.350399 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 261) took 1096 ms. Trace:
I20260809 01:45:51.350831 12222 rpcz_store.cc:276] 0809 01:45:50.254122 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:50.254235 (+ 113us) service_pool.cc:224] Handling call
0809 01:45:51.087086 (+832851us) ts_tablet_manager.cc:1151] Deleting tablet c16897857df24c6da5744a99d5f687c0
0809 01:45:51.087123 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:51.237231 (+150108us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:51.268654 (+ 31423us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:51.350213 (+ 81559us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":832325}
I20260809 01:45:51.314723 12700 tablet.cc:1724] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:51.360632 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36a542c773b7438ead16c5b3af224cf9" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:51.362803 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 5af35cf31a66474cb0575b2fb1547616 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:51.363385 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d35f291c92a45899eed0f2f10a3a1d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:51.364593 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet dc230f25baf94cbf80aa8e2771ec23ba with delete_type TABLET_DATA_DELETED (replaced by tablet 19fe81e277cd4c798a07ceafa0b60566) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:51.366892 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet c2f86bba4bd94fbcacdc9c139e9de2d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:51.380646 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f86bba4bd94fbcacdc9c139e9de2d7" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:51.382951 11383 raft_consensus.cc:2468] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:51.333496 11118 tablet_service.cc:1511] Processing CreateTablet for tablet b095338eb59e4161aaaf3afb762cd296 (DEFAULT_TABLE table=default.Table_0908ea7070134c749de4ff154f23628e [id=148da762500547349a600027071299ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.360491 11551 tablet_bootstrap.cc:654] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:51.390995 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b095338eb59e4161aaaf3afb762cd296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.392921 11105 tablet_service.cc:1511] Processing CreateTablet for tablet efe4c7b15604442eb7be83fe19bb54f4 (DEFAULT_TABLE table=default.Table_5da7b5860d5d457c8fa8e67d7f9833d5 [id=f9eab70ead9646b8a8b3c93c7b18d9dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.394235 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efe4c7b15604442eb7be83fe19bb54f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.394268 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet c16897857df24c6da5744a99d5f687c0 (table default.Table_6d4e0b3afc864450bdf1a8fb35f7dc75 [id=9b307bc98a124b07a2f46fafb20ce1f0]) successfully deleted
I20260809 01:45:51.395512 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 262) took 1140 ms. Trace:
I20260809 01:45:51.395993 12222 rpcz_store.cc:276] 0809 01:45:50.255308 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:50.255418 (+ 110us) service_pool.cc:224] Handling call
0809 01:45:51.395060 (+1139642us) ts_tablet_manager.cc:1151] Deleting tablet a38bd344108649ed8e1687f82154000b
0809 01:45:51.395116 (+ 56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:51.395446 (+ 330us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1139080}
W20260809 01:45:51.397269 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet a38bd344108649ed8e1687f82154000b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a38bd344108649ed8e1687f82154000b already in progress: creating tablet
I20260809 01:45:51.398241 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 263) took 1141 ms. Trace:
I20260809 01:45:51.401613 12222 rpcz_store.cc:276] 0809 01:45:50.256512 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:50.256616 (+ 104us) service_pool.cc:224] Handling call
0809 01:45:51.397964 (+1141348us) ts_tablet_manager.cc:1151] Deleting tablet ebc209c696ac4c669961f5ed8392af40
0809 01:45:51.398002 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:51.398195 (+ 193us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1140852}
W20260809 01:45:51.404091 11062 leader_election.cc:343] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:51.407219 11062 leader_election.cc:304] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:51.407812 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet ebc209c696ac4c669961f5ed8392af40 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ebc209c696ac4c669961f5ed8392af40 already in progress: creating tablet
I20260809 01:45:51.408380 11062 leader_election.cc:304] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:51.409217 12599 raft_consensus.cc:2749] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:51.409809 12599 raft_consensus.cc:2804] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:51.410101 12599 raft_consensus.cc:493] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:51.410392 12599 raft_consensus.cc:3060] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:51.410456 12222 tablet_replica.cc:333] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:51.412103 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d356fcc48441ca80f2df29b533404f" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:51.433859 12599 raft_consensus.cc:515] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.438274 11353 tablet_service.cc:1511] Processing CreateTablet for tablet d18d1f8343ed4c33a70fccbecc3012a6 (DEFAULT_TABLE table=default.Table_ae40e284667c40089594f8328c9630d7 [id=a510e3ee842f43d8931fe23f018bb7ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.439623 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d18d1f8343ed4c33a70fccbecc3012a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.447589 12545 leader_election.cc:290] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:51.448695 12654 leader_election.cc:290] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:51.453368 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_57b8282c32f843b1882c94f5479bd35a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:51.455461 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_57b8282c32f843b1882c94f5479bd35a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:51.461217 10586 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 991e8580b8624f34881d56c6cc60600f
W20260809 01:45:51.467376 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 197043us
I20260809 01:45:51.467723 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_e755d806f33849768faa61b4cc03b181"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:51.469763 10805 leader_election.cc:343] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:51.470202 10805 leader_election.cc:304] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:51.472023 12654 raft_consensus.cc:2749] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:51.468000 11553 raft_consensus.cc:359] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:51.474715 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e755d806f33849768faa61b4cc03b181 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:51.474946 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d5a709cd86476594613e788a325094" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:51.475662 11553 raft_consensus.cc:385] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:51.478358 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93334cc766044e79d37f293bc8556fb" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:51.479010 11553 raft_consensus.cc:740] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:51.486443 12710 consensus_queue.cc:1048] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 01:45:51.490834 10805 leader_election.cc:343] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:51.491314 10873 tablet_service.cc:1558] Processing DeleteTablet for tablet 991e8580b8624f34881d56c6cc60600f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:51 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:51.491266 10805 leader_election.cc:304] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:51.494190 11553 consensus_queue.cc:260] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.506307 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet ead75d766d0346f6823074d90ba8054e (table default.Table_3c356f5b797347c1ab98aaca3d06f4eb [id=028331846d434fc1beff5b11e2129233]) successfully deleted
I20260809 01:45:51.491688 11366 tablet_service.cc:1511] Processing CreateTablet for tablet b095338eb59e4161aaaf3afb762cd296 (DEFAULT_TABLE table=default.Table_0908ea7070134c749de4ff154f23628e [id=148da762500547349a600027071299ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.507658 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b095338eb59e4161aaaf3afb762cd296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.491453 12545 raft_consensus.cc:493] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:51.514864 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet 991e8580b8624f34881d56c6cc60600f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:51 UTC) from {username='slave'} at 127.0.0.1:53476
W20260809 01:45:51.496658 11289 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.10.8.124:33929, user_credentials={real_user=slave}} blocked reactor thread for 91761.2us
I20260809 01:45:51.502450 11361 tablet_service.cc:1511] Processing CreateTablet for tablet efe4c7b15604442eb7be83fe19bb54f4 (DEFAULT_TABLE table=default.Table_5da7b5860d5d457c8fa8e67d7f9833d5 [id=f9eab70ead9646b8a8b3c93c7b18d9dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.525180 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efe4c7b15604442eb7be83fe19bb54f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.465085 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet 991e8580b8624f34881d56c6cc60600f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:51 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:51.488052 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93334cc766044e79d37f293bc8556fb" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:51.530673 10858 tablet_service.cc:1511] Processing CreateTablet for tablet b095338eb59e4161aaaf3afb762cd296 (DEFAULT_TABLE table=default.Table_0908ea7070134c749de4ff154f23628e [id=148da762500547349a600027071299ab]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:51.530967 10804 leader_election.cc:343] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:51.531682 12222 raft_consensus.cc:2243] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:51.533334 12222 raft_consensus.cc:2272] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:51.540899 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 132591us
W20260809 01:45:51.542989 11062 leader_election.cc:343] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:51.535532 12649 raft_consensus.cc:2749] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:51.552059 10852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afafd99138c54fc6aa76c0cda0901842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.552897 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d18d1f8343ed4c33a70fccbecc3012a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.557372 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b095338eb59e4161aaaf3afb762cd296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.560554 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f86bba4bd94fbcacdc9c139e9de2d7" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:51.561218 11383 raft_consensus.cc:3060] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:51.548705 11063 leader_election.cc:343] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: c2f86bba4bd94fbcacdc9c139e9de2d7
I20260809 01:45:51.562534 11062 leader_election.cc:304] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:51.574147 10873 tablet_service.cc:1511] Processing CreateTablet for tablet efe4c7b15604442eb7be83fe19bb54f4 (DEFAULT_TABLE table=default.Table_5da7b5860d5d457c8fa8e67d7f9833d5 [id=f9eab70ead9646b8a8b3c93c7b18d9dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.535437 12545 raft_consensus.cc:515] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.575583 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efe4c7b15604442eb7be83fe19bb54f4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:51.577862 10576 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet e5af40994c204c4f82e36a53e00f2dfe (table default.Table_15c9a494eb954149a4095ebfca7505a6 [id=e995a84f2635461ca81f05d2ece827c6], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e5af40994c204c4f82e36a53e00f2dfe consensus state has no leader
I20260809 01:45:51.584975 12545 leader_election.cc:290] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:51.590315 11554 tablet_bootstrap.cc:654] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:51.611186 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 68228.7us
I20260809 01:45:51.612426 12720 raft_consensus.cc:2749] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:51.554378 12719 tablet.cc:1724] T 7df6343423604a00a46f203f61e98e2f P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:51.585629 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b31998b65e4264b16c0d73dbcfc73b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:51.614831 10804 leader_election.cc:343] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:51.618742 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 980adb971a594ad5bb6b1f8fce145b2f (DEFAULT_TABLE table=default.Table_aed544389a6747a8aea8e092dbbf896d [id=fda9cbfdb9f74020be7e8d2f22cedab3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.620298 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 980adb971a594ad5bb6b1f8fce145b2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.622349 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 7634fe763a4e4fec91c5938e45a5d1fc
I20260809 01:45:51.624081 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet 7634fe763a4e4fec91c5938e45a5d1fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:51 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:51.627272 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 7634fe763a4e4fec91c5938e45a5d1fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:51 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:51.630532 10586 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]). Expected version 3 got 0
I20260809 01:45:51.647639 12599 leader_election.cc:290] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:51.649919 12222 ts_tablet_manager.cc:1916] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:51.660679 10856 consensus_meta.cc:287] T 241ce7d4549848abab373eb22e9aba97 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent flushing consensus metadata: real 0.593s user 0.007s sys 0.002s
W20260809 01:45:51.663843 10864 consensus_meta.cc:287] T fe4afb4101ec445e94210e3722725b59 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent flushing consensus metadata: real 0.603s user 0.007s sys 0.003s
W20260809 01:45:51.677419 12726 log.cc:927] Time spent T cef88ffe9e95437da81793795742def3 P 33b5652538e64ed58a09993d3188f8d7: Append to log took a long time: real 0.113s user 0.001s sys 0.000s
I20260809 01:45:51.692307 11949 ts_tablet_manager.cc:1929] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:51.692998 11949 log.cc:1199] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/5af35cf31a66474cb0575b2fb1547616
I20260809 01:45:51.702093 11116 tablet_service.cc:1511] Processing CreateTablet for tablet 4ed45a2d5e9b4ad8be6f94e4a895003f (DEFAULT_TABLE table=default.Table_e39314f303ee429f897d4f85b8edbf69 [id=21bc1ee96b724341904d6b0c5b10fb7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.702080 11115 tablet_service.cc:1511] Processing CreateTablet for tablet 980adb971a594ad5bb6b1f8fce145b2f (DEFAULT_TABLE table=default.Table_aed544389a6747a8aea8e092dbbf896d [id=fda9cbfdb9f74020be7e8d2f22cedab3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.704514 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ed45a2d5e9b4ad8be6f94e4a895003f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.711198 11551 tablet_bootstrap.cc:492] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:51.711995 11551 ts_tablet_manager.cc:1403] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.442s user 0.017s sys 0.052s
I20260809 01:45:51.718986 10845 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:45966 (request call id 43) took 1050 ms. Trace:
I20260809 01:45:51.719326 10845 rpcz_store.cc:276] 0809 01:45:50.644232 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:50.644444 (+ 212us) service_pool.cc:224] Handling call
0809 01:45:51.596459 (+952015us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b5400053054. stack: 00007fe5ab55ecf3 00007fe5ab55ebf0 00007fe5ab10c574 0000000000516de4 0000000000516d79 00007fe5b265f951 00007fe5b26f02d8 00007fe5b375f4ed 00007fe5b1eb1104 00007fe5b1eb1092 00007fe5b1eb0fc1 00007fe5b1eb0f3c 00007fe5b1eb01f2 00007fe5b03c107c 00007fe5b03c04b6 00007fe5b03bfe5f
0809 01:45:51.663028 (+ 66569us) spinlock_profiling.cc:243] Waited 6.11 ms on lock 0x7b20000dbef8. stack: 00007fe5ab55ecf3 00007fe5ab55ebf0 00007fe5ab10c574 0000000000516de4 0000000000516d79 00007fe5b24dad71 00007fe5b24fc83c 00007fe5b24cdd52 00007fe5b2b420f2 00007fe5b2b41e55 00007fe5b375f4c0 00007fe5b1eb1104 00007fe5b1eb1092 00007fe5b1eb0fc1 00007fe5b1eb0f3c 00007fe5b1eb01f2
0809 01:45:51.677207 (+ 14179us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b5400112954. stack: 00007fe5ab55ecf3 00007fe5ab55ebf0 00007fe5ab10c574 0000000000516de4 0000000000516d79 00007fe5b265f951 00007fe5b26f02d8 00007fe5b375f4ed 00007fe5b1eb1104 00007fe5b1eb1092 00007fe5b1eb0fc1 00007fe5b1eb0f3c 00007fe5b1eb01f2 00007fe5b03c107c 00007fe5b03c04b6 00007fe5b03bfe5f
0809 01:45:51.694438 (+ 17231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":893764992}
W20260809 01:45:51.722427 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 231610us
I20260809 01:45:51.731748 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_c770b2448fca4145918eed5aa2fbc2f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:51.734230 10596 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_14f169b63955470587dbbdaea713291d" } modify_external_catalogs: true
I20260809 01:45:51.734886 10596 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_14f169b63955470587dbbdaea713291d" } modify_external_catalogs: true
I20260809 01:45:51.736088 10846 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:45966 (request call id 42) took 1101 ms. Trace:
I20260809 01:45:51.736429 10846 rpcz_store.cc:276] 0809 01:45:50.634522 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:50.634631 (+ 109us) service_pool.cc:224] Handling call
0809 01:45:51.686114 (+1051483us) spinlock_profiling.cc:243] Waited 38.2 ms on lock 0x7b5400112954. stack: 00007fe5ab55ecf3 00007fe5ab55ebf0 00007fe5ab10c574 0000000000516de4 0000000000516d79 00007fe5b265f951 00007fe5b26f02d8 00007fe5b375f4ed 00007fe5b1eb1104 00007fe5b1eb1092 00007fe5b1eb0fc1 00007fe5b1eb0f3c 00007fe5b1eb01f2 00007fe5b03c107c 00007fe5b03c04b6 00007fe5b03bfe5f
0809 01:45:51.735368 (+ 49254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87759104}
W20260809 01:45:51.738839 10805 leader_election.cc:343] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:51.739264 10805 leader_election.cc:304] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:51.740082 12728 raft_consensus.cc:2749] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:51.741309 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c770b2448fca4145918eed5aa2fbc2f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:51.742683 12710 consensus_queue.cc:1048] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:51.743162 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b31998b65e4264b16c0d73dbcfc73b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:51.755405 11395 raft_consensus.cc:2468] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:51.766894 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 980adb971a594ad5bb6b1f8fce145b2f (DEFAULT_TABLE table=default.Table_aed544389a6747a8aea8e092dbbf896d [id=fda9cbfdb9f74020be7e8d2f22cedab3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.767868 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 980adb971a594ad5bb6b1f8fce145b2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.770532 11553 ts_tablet_manager.cc:1434] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.652s user 0.018s sys 0.028s
W20260809 01:45:51.767144 11346 consensus_meta.cc:287] T fe4afb4101ec445e94210e3722725b59 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.513s user 0.007s sys 0.000s
W20260809 01:45:51.770834 11553 ts_tablet_manager.cc:1467] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8: Tablet startup took 652ms
I20260809 01:45:51.774008 11951 ts_tablet_manager.cc:1929] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:51.774490 11951 log.cc:1199] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/4754f2e2a35f4a6a91fa333e2cf564e4
W20260809 01:45:51.774782 10805 leader_election.cc:343] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:51.775249 11951 ts_tablet_manager.cc:1950] T 4754f2e2a35f4a6a91fa333e2cf564e4 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:51.775305 10805 leader_election.cc:304] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:51.771181 11553 ts_tablet_manager.cc:1469] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8: Trace:
0809 01:45:42.439364 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:42.439535 (+ 171us) service_pool.cc:224] Handling call
0809 01:45:42.480904 (+ 41369us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b5800000f98. stack: 00007fe5ab55ecf3 00007fe5ab55ebf0 00007fe5ab10c574 0000000000516de4 0000000000516d79 00007fe5ab419981 00007fe5ab55984c 00007fe5ab55940b 00007fe5b449dfb9 00007fe5b36df640 00007fe5b37c4641 00007fe5b37a18d3 00007fe5b3750210 00007fe5b1cdcbf4 00007fe5b1cdcb82 00007fe5b1cdcab1
0809 01:45:42.505600 (+ 24696us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:42.505732 (+ 132us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:42.831787 (+326055us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:43.220918 (+389131us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:43.221371 (+ 453us) inbound_call.cc:177] Queueing success response
0809 01:45:50.951770 (+7730399us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0809 01:45:51.118421 (+166651us) ts_tablet_manager.cc:1435] Starting tablet replica
0809 01:45:51.119376 (+ 955us) tablet_replica.cc:261] Starting instrumentation
0809 01:45:51.119834 (+ 458us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":55362,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":296,"spinlock_wait_cycles":5846656,"tablet-open.queue_time_us":7730499,"thread_start_us":256777,"threads_started":1}
I20260809 01:45:51.777661 10588 master_service.cc:960] ReplaceTablet: received request to replace tablet 750aa437ef594587a512a9c9c8278aa8 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:51.778553 11553 tablet_bootstrap.cc:492] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:51.780942 12727 raft_consensus.cc:2749] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:51.782068 11062 leader_election.cc:304] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:51.782752 11062 raft_consensus.cc:2686] T c16897857df24c6da5744a99d5f687c0 P 33b5652538e64ed58a09993d3188f8d7: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 01:45:51.782454 12727 consensus_queue.cc:1048] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:51.785960 12723 consensus_queue.cc:1048] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:51.793457 10873 tablet_service.cc:1511] Processing CreateTablet for tablet 4ed45a2d5e9b4ad8be6f94e4a895003f (DEFAULT_TABLE table=default.Table_e39314f303ee429f897d4f85b8edbf69 [id=21bc1ee96b724341904d6b0c5b10fb7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.794445 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ed45a2d5e9b4ad8be6f94e4a895003f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.801687 11949 ts_tablet_manager.cc:1950] T 5af35cf31a66474cb0575b2fb1547616 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:51.804726 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 238) took 5097 ms. Trace:
I20260809 01:45:51.807890 12222 ts_tablet_manager.cc:1929] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:51.808434 12222 log.cc:1199] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/c2f86bba4bd94fbcacdc9c139e9de2d7
I20260809 01:45:51.809257 12222 ts_tablet_manager.cc:1950] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:51.830354 10597 catalog_manager.cc:5719] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65). New cstate: current_term: 1 leader_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:51.831578 10597 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]). Expected version 3 got 0
I20260809 01:45:51.832597 10597 catalog_manager.cc:5719] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65). New cstate: current_term: 1 leader_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: HEALTHY } } }
I20260809 01:45:51.836031 10588 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_11fb3bbd4a124fcf86837c922880b598" } modify_external_catalogs: true
I20260809 01:45:51.836508 10588 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_11fb3bbd4a124fcf86837c922880b598" } modify_external_catalogs: true
I20260809 01:45:51.839424 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet 7634fe763a4e4fec91c5938e45a5d1fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:51 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:51.846393 10589 master_service.cc:960] ReplaceTablet: received request to replace tablet d146bdaf4dd2418ca870c4afeb6ccb6c from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:51.806226 12723 raft_consensus.cc:493] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:51.817318 12728 raft_consensus.cc:493] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:51.831740 11553 tablet_bootstrap.cc:654] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:51.925971 12723 raft_consensus.cc:515] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.926055 12728 raft_consensus.cc:515] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.843297 10873 tablet_service.cc:1511] Processing CreateTablet for tablet e3690212d43543fc982815dcff225a51 (DEFAULT_TABLE table=default.Table_3c356f5b797347c1ab98aaca3d06f4eb [id=028331846d434fc1beff5b11e2129233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.928112 12723 leader_election.cc:290] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:51.928341 12728 leader_election.cc:290] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:51.928828 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3690212d43543fc982815dcff225a51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.905577 10852 tablet_service.cc:1511] Processing CreateTablet for tablet c5336aa032eb44269bbd7e407782a072 (DEFAULT_TABLE table=default.Table_da48f5f78afb48f09e10d02abccf6cd8 [id=9d439c85b53742c790d0f59ff36f6733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.934846 10852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5336aa032eb44269bbd7e407782a072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.835212 11115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 980adb971a594ad5bb6b1f8fce145b2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.842397 11105 tablet_service.cc:1558] Processing DeleteTablet for tablet b84f1943c1a34e999fc7c12a5847cdb6 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ed45a2d5e9b4ad8be6f94e4a895003f) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:51.854087 11124 tablet_service.cc:1511] Processing CreateTablet for tablet e3690212d43543fc982815dcff225a51 (DEFAULT_TABLE table=default.Table_3c356f5b797347c1ab98aaca3d06f4eb [id=028331846d434fc1beff5b11e2129233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.910435 10855 tablet_service.cc:1511] Processing CreateTablet for tablet 6dd979cdbf48493581caf81a29947344 (DEFAULT_TABLE table=default.Table_cf4d3615b483433d81b9cb61a945a25e [id=37ec9a8b5ec74d7fbd8bd053d283a573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.933687 10589 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_2ee9bb5a08bb43278f67198743327a71" } new_table_name: "default.Table_3c6e740f36be424d8517db3df713ddd2" modify_external_catalogs: true
I20260809 01:45:51.937749 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3690212d43543fc982815dcff225a51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.938015 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd979cdbf48493581caf81a29947344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.866376 12599 raft_consensus.cc:493] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:51.941735 12599 raft_consensus.cc:515] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.943660 12599 leader_election.cc:290] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:51.867683 12720 consensus_queue.cc:1048] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:51.878196 11551 raft_consensus.cc:359] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.944693 11551 raft_consensus.cc:385] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:51.944952 11551 raft_consensus.cc:740] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:51.945497 11551 consensus_queue.cc:260] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.946849 11551 ts_tablet_manager.cc:1434] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.234s user 0.001s sys 0.005s
I20260809 01:45:51.949942 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e4315b3c644846ac572883c51f23de" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:51.950451 10910 raft_consensus.cc:2468] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:51.950762 11347 tablet_service.cc:1511] Processing CreateTablet for tablet e3690212d43543fc982815dcff225a51 (DEFAULT_TABLE table=default.Table_3c356f5b797347c1ab98aaca3d06f4eb [id=028331846d434fc1beff5b11e2129233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.951736 11347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3690212d43543fc982815dcff225a51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.904166 11118 tablet_service.cc:1511] Processing CreateTablet for tablet c5336aa032eb44269bbd7e407782a072 (DEFAULT_TABLE table=default.Table_da48f5f78afb48f09e10d02abccf6cd8 [id=9d439c85b53742c790d0f59ff36f6733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.952874 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5336aa032eb44269bbd7e407782a072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.953190 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 237) took 4502 ms. Trace:
I20260809 01:45:51.953493 11951 rpcz_store.cc:276] 0809 01:45:47.451020 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:47.451169 (+ 149us) service_pool.cc:224] Handling call
0809 01:45:51.033023 (+3581854us) ts_tablet_manager.cc:1151] Deleting tablet 4754f2e2a35f4a6a91fa333e2cf564e4
0809 01:45:51.033083 (+ 60us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:51.360462 (+327379us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:51.773877 (+413415us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:51.953130 (+179253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":175709,"tablet-delete.queue_time_us":3581113}
I20260809 01:45:51.954905 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 246) took 4262 ms. Trace:
I20260809 01:45:51.955191 11951 rpcz_store.cc:276] 0809 01:45:47.692891 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:47.693234 (+ 343us) service_pool.cc:224] Handling call
0809 01:45:51.954585 (+4261351us) ts_tablet_manager.cc:1151] Deleting tablet 977694496e014ef29f5d7d513e5219c0
0809 01:45:51.954627 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:51.954855 (+ 228us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4260445}
I20260809 01:45:51.956246 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 247) took 4251 ms. Trace:
I20260809 01:45:51.956522 11951 rpcz_store.cc:276] 0809 01:45:47.704909 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:47.705030 (+ 121us) service_pool.cc:224] Handling call
0809 01:45:51.955959 (+4250929us) ts_tablet_manager.cc:1151] Deleting tablet 19fe81e277cd4c798a07ceafa0b60566
0809 01:45:51.955996 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:51.956201 (+ 205us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4250407}
I20260809 01:45:51.929507 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19fe81e277cd4c798a07ceafa0b60566" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:51.930190 11154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c73bed7a9774cbc8ac774e6514a981d" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:51.959262 11063 leader_election.cc:304] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:51.959937 12720 raft_consensus.cc:2804] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:51.960245 12720 raft_consensus.cc:493] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:51.960577 12720 raft_consensus.cc:3060] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:51.962483 11361 tablet_service.cc:1511] Processing CreateTablet for tablet c5336aa032eb44269bbd7e407782a072 (DEFAULT_TABLE table=default.Table_da48f5f78afb48f09e10d02abccf6cd8 [id=9d439c85b53742c790d0f59ff36f6733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:51.963454 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5336aa032eb44269bbd7e407782a072. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:51.963847 10804 leader_election.cc:343] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:51.964128 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 4754f2e2a35f4a6a91fa333e2cf564e4 (table default.Table_d1d088df5e7d452ab5a8771a8d3679dd [id=bd9dd208af1e4bd596ae7e3a4b219faf]) successfully deleted
W20260809 01:45:51.964861 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 977694496e014ef29f5d7d513e5219c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 977694496e014ef29f5d7d513e5219c0 already in progress: creating tablet
W20260809 01:45:51.964919 10804 leader_election.cc:343] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:51.965471 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 19fe81e277cd4c798a07ceafa0b60566 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 19fe81e277cd4c798a07ceafa0b60566 already in progress: creating tablet
I20260809 01:45:51.967615 12720 raft_consensus.cc:515] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:51.971094 12720 leader_election.cc:290] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:51.972612 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e4315b3c644846ac572883c51f23de" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:51.973107 10910 raft_consensus.cc:3060] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:51.990654 10910 raft_consensus.cc:2468] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:51.997022 11063 leader_election.cc:304] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:51.998588 12599 raft_consensus.cc:2804] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:51.999126 12599 raft_consensus.cc:697] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:52.000427 12599 consensus_queue.cc:237] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:52.002267 10592 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3e03033672e947eab8cc143263281133 consensus state has no leader
I20260809 01:45:52.008692 11551 tablet_bootstrap.cc:492] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:52.010167 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 264) took 1748 ms. Trace:
I20260809 01:45:52.010586 12222 rpcz_store.cc:276] 0809 01:45:50.262229 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:50.262696 (+ 467us) service_pool.cc:224] Handling call
0809 01:45:51.410272 (+1147576us) ts_tablet_manager.cc:1151] Deleting tablet c2f86bba4bd94fbcacdc9c139e9de2d7
0809 01:45:51.410331 (+ 59us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:51.533118 (+122787us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54000933d4. stack: 00007fe4bc3e2cf3 00007fe4bc3e2bf0 00007fe4bbf90574 0000000000516de4 0000000000516d79 00007fe4c34e3951 00007fe4c3582742 00007fe4c39c2d2c 00007fe4c462a41a 00007fe4c465b0e0 00007fe4c464164d 00007fe4c46415f9 00007fe4c4641589 00007fe4c4641551 00007fe4c464077d 00000000005bd8c4
0809 01:45:51.663782 (+130664us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:51.807731 (+143949us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.010095 (+202364us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72761,"spinlock_wait_cycles":276563200,"tablet-delete.queue_time_us":1147054}
I20260809 01:45:52.012089 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet c2f86bba4bd94fbcacdc9c139e9de2d7 (table default.Table_5f4d617a86c2487fa9f64cb58f14de3c [id=f64daa5a2706436081e9166a5f3a0cd7]) successfully deleted
I20260809 01:45:52.013304 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 271) took 1097 ms. Trace:
I20260809 01:45:51.928551 11949 rpcz_store.cc:276] 0809 01:45:46.707662 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:46.709642 (+ 1980us) service_pool.cc:224] Handling call
0809 01:45:50.677440 (+3967798us) ts_tablet_manager.cc:1151] Deleting tablet 5af35cf31a66474cb0575b2fb1547616
0809 01:45:50.680764 (+ 3324us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:51.039386 (+358622us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:51.692151 (+652765us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:51.804649 (+112498us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3967140}
I20260809 01:45:52.015514 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 5af35cf31a66474cb0575b2fb1547616 (table default.Table_8031d30fd4be45869a2e535707f3459b [id=ab949c442b89438cbd00a59630ab23e3]) successfully deleted
I20260809 01:45:52.013659 12222 rpcz_store.cc:276] 0809 01:45:50.915767 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:50.924613 (+ 8846us) service_pool.cc:224] Handling call
0809 01:45:52.012911 (+1088298us) ts_tablet_manager.cc:1151] Deleting tablet 1c73bed7a9774cbc8ac774e6514a981d
0809 01:45:52.012952 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.013236 (+ 284us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1087670}
I20260809 01:45:52.032691 12222 tablet_replica.cc:333] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:52.033464 12222 raft_consensus.cc:2243] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:52.034018 12222 raft_consensus.cc:2272] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:52.035993 12222 ts_tablet_manager.cc:1916] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:52.048050 11105 tablet_service.cc:1511] Processing CreateTablet for tablet 6dd979cdbf48493581caf81a29947344 (DEFAULT_TABLE table=default.Table_cf4d3615b483433d81b9cb61a945a25e [id=37ec9a8b5ec74d7fbd8bd053d283a573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:52.049006 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd979cdbf48493581caf81a29947344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:52.048889 12728 consensus_queue.cc:1048] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:52.051955 11111 tablet_service.cc:1511] Processing CreateTablet for tablet 6680c7d6379b43bf8b31e610ddc824d5 (DEFAULT_TABLE table=default.Table_2ee9bb5a08bb43278f67198743327a71 [id=71cdf2ceeb8c4e4d9635023c1c11c428]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:52.057827 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 1c73bed7a9774cbc8ac774e6514a981d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c73bed7a9774cbc8ac774e6514a981d already in progress: creating tablet
I20260809 01:45:52.063454 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 6dd979cdbf48493581caf81a29947344 (DEFAULT_TABLE table=default.Table_cf4d3615b483433d81b9cb61a945a25e [id=37ec9a8b5ec74d7fbd8bd053d283a573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:52.064623 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd979cdbf48493581caf81a29947344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:52.078318 11111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6680c7d6379b43bf8b31e610ddc824d5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:52.088073 10805 consensus_peers.cc:597] T 83ac8498dff04b4e93303d7a35e62134 P b5117038a52a45b0aea3d30ca9cfcbe8 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260809 01:45:51.978863 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 111517us
I20260809 01:45:52.003328 12742 consensus_queue.cc:1048] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.088s
I20260809 01:45:52.092684 12742 raft_consensus.cc:493] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:52.125144 10592 master_service.cc:960] ReplaceTablet: received request to replace tablet b9493892c1ff48e7a0558ff0fcb7a0f3 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:52.164435 12728 raft_consensus.cc:493] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:52.165055 12728 raft_consensus.cc:515] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.093294 12742 raft_consensus.cc:515] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.177954 11551 tablet_bootstrap.cc:654] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:52.078554 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19fe81e277cd4c798a07ceafa0b60566" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:52.081915 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c73bed7a9774cbc8ac774e6514a981d" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:52.094833 11346 tablet_service.cc:1511] Processing CreateTablet for tablet 6680c7d6379b43bf8b31e610ddc824d5 (DEFAULT_TABLE table=default.Table_2ee9bb5a08bb43278f67198743327a71 [id=71cdf2ceeb8c4e4d9635023c1c11c428]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:52.099109 11383 consensus_meta.cc:287] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.537s user 0.012s sys 0.000s
W20260809 01:45:52.198534 10805 leader_election.cc:343] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.198945 10805 leader_election.cc:304] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:52.198906 11383 raft_consensus.cc:2468] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:52.199779 12545 raft_consensus.cc:2749] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:52.200209 10805 leader_election.cc:343] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.200543 10805 leader_election.cc:304] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:52.108640 11337 tablet_service.cc:1511] Processing CreateTablet for tablet 4ed45a2d5e9b4ad8be6f94e4a895003f (DEFAULT_TABLE table=default.Table_e39314f303ee429f897d4f85b8edbf69 [id=21bc1ee96b724341904d6b0c5b10fb7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:52.154165 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 746dfe3a938248b0a44b01b59c431632 (DEFAULT_TABLE table=default.Table_5d441fff2d57479ca948ad1d62e2b660 [id=47cd517945324ea8ae814239238ce805]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:52.201134 12545 raft_consensus.cc:2749] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:52.202698 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746dfe3a938248b0a44b01b59c431632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:52.205873 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6680c7d6379b43bf8b31e610ddc824d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:52.207617 11337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ed45a2d5e9b4ad8be6f94e4a895003f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:51.957506 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 248) took 4241 ms. Trace:
I20260809 01:45:52.230649 11951 rpcz_store.cc:276] 0809 01:45:47.716267 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:47.716398 (+ 131us) service_pool.cc:224] Handling call
0809 01:45:51.957233 (+4240835us) ts_tablet_manager.cc:1151] Deleting tablet e93334cc766044e79d37f293bc8556fb
0809 01:45:51.957267 (+ 34us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:51.957465 (+ 198us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4240338}
W20260809 01:45:52.232163 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet e93334cc766044e79d37f293bc8556fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e93334cc766044e79d37f293bc8556fb already in progress: creating tablet
I20260809 01:45:52.233035 11951 tablet_replica.cc:333] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:52.233805 11951 raft_consensus.cc:2243] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:52.236923 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e4315b3c644846ac572883c51f23de" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:52.187999 12700 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:53476 (request call id 268) took 1883 ms. Trace:
I20260809 01:45:52.260792 11127 tablet_service.cc:1511] Processing CreateTablet for tablet f1d5fdaaed9b40ceab053797439a38c9 (DEFAULT_TABLE table=default.Table_4f9017aa8338441e84bc6913c4915e42 [id=1c059f33595f4ec3a5fa72d085282765]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:52.261641 12734 log.cc:927] Time spent T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7: Append to log took a long time: real 0.074s user 0.000s sys 0.000s
I20260809 01:45:52.261691 10877 tablet_service.cc:1511] Processing CreateTablet for tablet 6680c7d6379b43bf8b31e610ddc824d5 (DEFAULT_TABLE table=default.Table_2ee9bb5a08bb43278f67198743327a71 [id=71cdf2ceeb8c4e4d9635023c1c11c428]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:52.263918 11553 tablet_bootstrap.cc:492] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:52.264387 11553 ts_tablet_manager.cc:1403] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.486s user 0.010s sys 0.009s
I20260809 01:45:52.267215 11553 raft_consensus.cc:359] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.267817 11553 raft_consensus.cc:385] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:52.268090 11553 raft_consensus.cc:740] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:52.270085 11553 consensus_queue.cc:260] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.272071 11553 ts_tablet_manager.cc:1434] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260809 01:45:52.258538 11110 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 272) took 1225 ms. Trace:
I20260809 01:45:52.276120 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 746dfe3a938248b0a44b01b59c431632 (DEFAULT_TABLE table=default.Table_5d441fff2d57479ca948ad1d62e2b660 [id=47cd517945324ea8ae814239238ce805]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:52.279564 11062 leader_election.cc:304] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:52.280194 11062 raft_consensus.cc:2686] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 33b5652538e64ed58a09993d3188f8d7: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 01:45:52.283808 12728 leader_election.cc:290] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:52.294627 11113 tablet_service.cc:1511] Processing CreateTablet for tablet 746dfe3a938248b0a44b01b59c431632 (DEFAULT_TABLE table=default.Table_5d441fff2d57479ca948ad1d62e2b660 [id=47cd517945324ea8ae814239238ce805]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:52.295639 11949 tablet_replica.cc:333] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:52.296509 11949 raft_consensus.cc:2243] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:52.297070 11949 raft_consensus.cc:2272] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:52.297768 11951 raft_consensus.cc:2272] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:52.299711 11949 ts_tablet_manager.cc:1916] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:52.304987 10852 tablet_service.cc:1511] Processing CreateTablet for tablet f1d5fdaaed9b40ceab053797439a38c9 (DEFAULT_TABLE table=default.Table_4f9017aa8338441e84bc6913c4915e42 [id=1c059f33595f4ec3a5fa72d085282765]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:52.312374 11125 tablet_service.cc:1511] Processing CreateTablet for tablet e9b96ed2801d4064a88bb99243debd28 (DEFAULT_TABLE table=default.Table_706bcc86a9534c99b53b27593052f412 [id=dcb1e716d7924acea67cfa7bd18c3aef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:52.277537 11110 rpcz_store.cc:276] 0809 01:45:51.010533 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.012220 (+ 1687us) service_pool.cc:224] Handling call
0809 01:45:51.035217 (+ 22997us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:51.035320 (+ 103us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:51.656589 (+621269us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.076472 (+419883us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:52.235897 (+159425us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93793}
W20260809 01:45:52.325110 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 45637.4us
I20260809 01:45:52.326066 10595 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:37956 (request call id 367) took 1160 ms. Trace:
W20260809 01:45:52.326465 11062 leader_election.cc:343] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.326426 10595 rpcz_store.cc:276] 0809 01:45:51.165566 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.169238 (+ 3672us) service_pool.cc:224] Handling call
0809 01:45:51.230276 (+ 61038us) catalog_manager.cc:2127] Acquired catalog manager lock
0809 01:45:51.232344 (+ 2068us) catalog_manager.cc:2173] Created new table and tablet info
0809 01:45:52.325291 (+1092947us) catalog_manager.cc:2233] Wrote table and tablets to system table
0809 01:45:52.325726 (+ 435us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0809 01:45:52.326010 (+ 284us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:51.871725 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:51.873117 (+ 1392us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:51.873144 (+ 27us) write_op.cc:435] Acquired schema lock
0809 01:45:51.873155 (+ 11us) tablet.cc:662] Decoding operations
0809 01:45:51.873416 (+ 261us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:51.873447 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:51.873467 (+ 20us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:51.873667 (+ 200us) tablet.cc:701] Row locks acquired
0809 01:45:51.873679 (+ 12us) write_op.cc:260] PREPARE: finished
0809 01:45:51.873775 (+ 96us) write_op.cc:270] Start()
0809 01:45:51.873853 (+ 78us) write_op.cc:276] Timestamp: P: 1786239951873703 usec, L: 0
0809 01:45:51.873867 (+ 14us) op_driver.cc:348] REPLICATION: starting
0809 01:45:51.874261 (+ 394us) log.cc:844] Serialized 433 byte log entry
0809 01:45:52.321562 (+447301us) op_driver.cc:464] REPLICATION: finished
0809 01:45:52.322746 (+ 1184us) write_op.cc:301] APPLY: starting
0809 01:45:52.322817 (+ 71us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:52.322951 (+ 134us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:52.323131 (+ 180us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:52.323942 (+ 811us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:52.324209 (+ 267us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 01:45:52.324301 (+ 92us) write_op.cc:312] APPLY: finished
0809 01:45:52.324638 (+ 337us) log.cc:844] Serialized 29 byte log entry
0809 01:45:52.324907 (+ 269us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:52.325003 (+ 96us) write_op.cc:454] Released schema lock
0809 01:45:52.325039 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:52.325448 (+ 409us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:52.325670 (+ 222us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":995,"apply.run_cpu_time_us":2601,"apply.run_wall_time_us":2600,"num_ops":2,"prepare.queue_time_us":520337,"prepare.run_cpu_time_us":1690,"prepare.run_wall_time_us":2722,"replication_time_us":447574}]]}
I20260809 01:45:52.339378 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e4315b3c644846ac572883c51f23de" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:52.345798 11354 tablet_service.cc:1511] Processing CreateTablet for tablet f1d5fdaaed9b40ceab053797439a38c9 (DEFAULT_TABLE table=default.Table_4f9017aa8338441e84bc6913c4915e42 [id=1c059f33595f4ec3a5fa72d085282765]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:52.347787 11341 tablet_service.cc:1511] Processing CreateTablet for tablet e9b96ed2801d4064a88bb99243debd28 (DEFAULT_TABLE table=default.Table_706bcc86a9534c99b53b27593052f412 [id=dcb1e716d7924acea67cfa7bd18c3aef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:52.349339 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a238ad3d46054daa9435e9076ba8311d" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:52.354194 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1d5fdaaed9b40ceab053797439a38c9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:52.361629 10805 leader_election.cc:343] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.361989 11553 tablet_bootstrap.cc:492] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:52.361390 10852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1d5fdaaed9b40ceab053797439a38c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:52.366384 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746dfe3a938248b0a44b01b59c431632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:52.368952 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6680c7d6379b43bf8b31e610ddc824d5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:52.370769 10581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.125:42063, user_credentials={real_user=slave}} blocked reactor thread for 59093.6us
I20260809 01:45:52.371830 10873 tablet_service.cc:1511] Processing CreateTablet for tablet e9b96ed2801d4064a88bb99243debd28 (DEFAULT_TABLE table=default.Table_706bcc86a9534c99b53b27593052f412 [id=dcb1e716d7924acea67cfa7bd18c3aef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:52.373482 12545 raft_consensus.cc:493] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:52.374259 11289 consensus_peers.cc:597] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6d35f291c92a45899eed0f2f10a3a1d8. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:52.374136 12545 raft_consensus.cc:515] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.378006 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b96ed2801d4064a88bb99243debd28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:52.378098 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b96ed2801d4064a88bb99243debd28. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:52.393599 11283 consensus_peers.cc:597] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6d35f291c92a45899eed0f2f10a3a1d8. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:52.395056 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc9c8e0f97e41b48a919aa579ad85cd" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:52.396020 11553 tablet_bootstrap.cc:654] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:52.399585 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a238ad3d46054daa9435e9076ba8311d" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:52.397310 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 47e8e02c848d46f48d44b0de414b666d (DEFAULT_TABLE table=default.Table_2b48e5de5f194f2caa53cdd253a8eda4 [id=27956a05763844f8b8b471089c672e20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:52.400947 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1d5fdaaed9b40ceab053797439a38c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:52.405855 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e8e02c848d46f48d44b0de414b666d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:52.407447 11110 tablet_service.cc:1511] Processing CreateTablet for tablet 47e8e02c848d46f48d44b0de414b666d (DEFAULT_TABLE table=default.Table_2b48e5de5f194f2caa53cdd253a8eda4 [id=27956a05763844f8b8b471089c672e20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:52.408543 11110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e8e02c848d46f48d44b0de414b666d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:52.410267 11125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b96ed2801d4064a88bb99243debd28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:52.414635 11113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746dfe3a938248b0a44b01b59c431632. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:52.418696 11062 leader_election.cc:343] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.426354 12723 raft_consensus.cc:493] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:52.426893 12723 raft_consensus.cc:515] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:52.434628 12723 leader_election.cc:290] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:52.446795 11351 consensus_meta.cc:287] T 9a354a7cc2a84123a456125ef6c9828a P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.851s user 0.002s sys 0.011s
I20260809 01:45:52.451484 11554 tablet_bootstrap.cc:492] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:52.452688 11554 ts_tablet_manager.cc:1403] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 1.434s user 0.024s sys 0.003s
I20260809 01:45:52.252031 12700 rpcz_store.cc:276] 0809 01:45:50.304116 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:50.304246 (+ 130us) service_pool.cc:224] Handling call
0809 01:45:52.187922 (+1883676us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:50.305726 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0809 01:45:50.392719 (+ 86993us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 01:45:50.392761 (+ 42us) alter_schema_op.cc:66] Acquired schema lock
0809 01:45:50.392834 (+ 73us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0809 01:45:50.393050 (+ 216us) alter_schema_op.cc:128] START. Timestamp: 1786239950392892
0809 01:45:50.393096 (+ 46us) op_driver.cc:348] REPLICATION: starting
0809 01:45:50.590156 (+197060us) log.cc:844] Serialized 173 byte log entry
0809 01:45:50.591191 (+ 1035us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54000a2b54. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee901f951 00007f0ee90b2208 00007f0ee9541f61 00007f0ee9541738 00007f0ee95463d1 00007f0ee9546389 00007f0ee9546319 00007f0ee95462e1 00007f0ee954550d 00000000005bd8c4 00000000005bd6f9
0809 01:45:50.844533 (+253342us) op_driver.cc:464] REPLICATION: finished
0809 01:45:51.314460 (+469927us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0809 01:45:51.355166 (+ 40706us) log.cc:844] Serialized 5 byte log entry
0809 01:45:52.124518 (+769352us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.186724 (+ 62206us) log.cc:844] Serialized 14 byte log entry
0809 01:45:52.187713 (+ 989us) alter_schema_op.cc:73] Released schema lock
0809 01:45:52.187789 (+ 76us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":469670,"prepare.queue_time_us":601,"prepare.run_cpu_time_us":6772,"prepare.run_wall_time_us":285596,"replication_time_us":448313,"spinlock_wait_cycles":451479424,"thread_start_us":185652,"threads_started":1}]]}
W20260809 01:45:52.460433 11063 leader_election.cc:343] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: b84f1943c1a34e999fc7c12a5847cdb6
I20260809 01:45:52.462375 10598 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:37956 (request call id 371) took 1004 ms. Trace:
I20260809 01:45:52.461534 12700 tablet.cc:1724] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:52.462657 10598 rpcz_store.cc:276] 0809 01:45:51.458274 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.467187 (+ 8913us) service_pool.cc:224] Handling call
0809 01:45:51.475233 (+ 8046us) catalog_manager.cc:2127] Acquired catalog manager lock
0809 01:45:51.484091 (+ 8858us) catalog_manager.cc:2173] Created new table and tablet info
0809 01:45:52.461617 (+977526us) catalog_manager.cc:2233] Wrote table and tablets to system table
0809 01:45:52.461962 (+ 345us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0809 01:45:52.462317 (+ 355us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:51.876291 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:51.876605 (+ 314us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:51.876625 (+ 20us) write_op.cc:435] Acquired schema lock
0809 01:45:51.876637 (+ 12us) tablet.cc:662] Decoding operations
0809 01:45:51.876866 (+ 229us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:51.876895 (+ 29us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:51.876915 (+ 20us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:51.877071 (+ 156us) tablet.cc:701] Row locks acquired
0809 01:45:51.877083 (+ 12us) write_op.cc:260] PREPARE: finished
0809 01:45:51.877167 (+ 84us) write_op.cc:270] Start()
0809 01:45:51.882764 (+ 5597us) write_op.cc:276] Timestamp: P: 1786239951877099 usec, L: 0
0809 01:45:51.882795 (+ 31us) op_driver.cc:348] REPLICATION: starting
0809 01:45:51.883200 (+ 405us) log.cc:844] Serialized 433 byte log entry
0809 01:45:52.456020 (+572820us) op_driver.cc:464] REPLICATION: finished
0809 01:45:52.456776 (+ 756us) write_op.cc:301] APPLY: starting
0809 01:45:52.458244 (+ 1468us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:52.458467 (+ 223us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:52.458702 (+ 235us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:52.459641 (+ 939us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:52.459921 (+ 280us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 01:45:52.460007 (+ 86us) write_op.cc:312] APPLY: finished
0809 01:45:52.460274 (+ 267us) log.cc:844] Serialized 29 byte log entry
0809 01:45:52.461176 (+ 902us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:52.461286 (+ 110us) write_op.cc:454] Released schema lock
0809 01:45:52.461343 (+ 57us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":521,"num_ops":2,"prepare.queue_time_us":390733,"prepare.run_cpu_time_us":1584,"prepare.run_wall_time_us":7083,"replication_time_us":573059}]]}
W20260809 01:45:52.450196 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 52775.6us
I20260809 01:45:52.459365 11951 ts_tablet_manager.cc:1916] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:52.470878 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325e3d19b83545a197c2e09fc9643b5e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:52.472720 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc9c8e0f97e41b48a919aa579ad85cd" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:52.459841 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325e3d19b83545a197c2e09fc9643b5e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:52.478557 10586 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_18e8960a1b8a457fb622abd881d541ed [id=b7fa92e91f85479c82ba9d384c44d429] alter complete (version 1)
W20260809 01:45:52.483497 10586 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.67:40463 (request call id 54) took 944 ms (client timeout 1000 ms). Trace:
I20260809 01:45:52.486104 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 4754f2e2a35f4a6a91fa333e2cf564e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:38 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:52.487185 12743 raft_consensus.cc:493] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:52.484081 10586 rpcz_store.cc:269] 0809 01:45:51.538817 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.538948 (+ 131us) service_pool.cc:224] Handling call
0809 01:45:51.625740 (+ 86792us) spinlock_profiling.cc:243] Waited 83.1 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:52.483405 (+857665us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:51.883473 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:51.883820 (+ 347us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:51.883840 (+ 20us) write_op.cc:435] Acquired schema lock
0809 01:45:51.883851 (+ 11us) tablet.cc:662] Decoding operations
0809 01:45:51.884053 (+ 202us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:51.884083 (+ 30us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:51.884103 (+ 20us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:51.884217 (+ 114us) tablet.cc:701] Row locks acquired
0809 01:45:51.884229 (+ 12us) write_op.cc:260] PREPARE: finished
0809 01:45:51.884310 (+ 81us) write_op.cc:270] Start()
0809 01:45:51.884371 (+ 61us) write_op.cc:276] Timestamp: P: 1786239951884240 usec, L: 0
0809 01:45:51.884384 (+ 13us) op_driver.cc:348] REPLICATION: starting
0809 01:45:51.884778 (+ 394us) log.cc:844] Serialized 356 byte log entry
0809 01:45:52.456454 (+571676us) op_driver.cc:464] REPLICATION: finished
0809 01:45:52.466901 (+ 10447us) write_op.cc:301] APPLY: starting
0809 01:45:52.466986 (+ 85us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:52.467115 (+ 129us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:52.467288 (+ 173us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:52.467617 (+ 329us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:52.467873 (+ 256us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:52.467960 (+ 87us) write_op.cc:312] APPLY: finished
0809 01:45:52.468253 (+ 293us) log.cc:844] Serialized 23 byte log entry
0809 01:45:52.468473 (+ 220us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:52.468538 (+ 65us) write_op.cc:454] Released schema lock
0809 01:45:52.478156 (+ 9618us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":6,"spinlock_wait_cycles":191021824,"child_traces":[["op",{"apply.queue_time_us":10248,"num_ops":1,"prepare.queue_time_us":249926,"prepare.run_cpu_time_us":1485,"prepare.run_wall_time_us":1485,"replication_time_us":571931}]]}
I20260809 01:45:52.487644 12743 raft_consensus.cc:515] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.460427 12742 leader_election.cc:290] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:52.504495 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84f1943c1a34e999fc7c12a5847cdb6" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:52.505831 11551 tablet_bootstrap.cc:492] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:52.509127 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb9afd3be194830ba99f5e20fdaca0a" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:52.506583 12743 leader_election.cc:290] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:52.523847 11551 ts_tablet_manager.cc:1403] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.515s user 0.014s sys 0.011s
W20260809 01:45:52.470742 11155 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s user 0.026s sys 0.009s
W20260809 01:45:52.508600 11062 leader_election.cc:343] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.524564 11062 leader_election.cc:304] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:52.525290 11155 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s user 0.027s sys 0.009s
I20260809 01:45:52.529727 12743 raft_consensus.cc:2749] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:52.530543 11951 ts_tablet_manager.cc:1929] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
W20260809 01:45:52.531409 11062 leader_election.cc:343] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.533216 11951 log.cc:1199] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/72b4850de7d740d794283ca93152fdc1
I20260809 01:45:52.540282 11951 ts_tablet_manager.cc:1950] T 72b4850de7d740d794283ca93152fdc1 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:52.549933 11351 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 275) took 1457 ms. Trace:
I20260809 01:45:52.551781 11551 raft_consensus.cc:359] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.552430 11551 raft_consensus.cc:385] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:52.552686 11551 raft_consensus.cc:740] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:52.565130 10596 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet c373ab7f921e46a2842ede6258df83af
I20260809 01:45:52.571712 10588 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet d986a05de4a848d09141f7e9794e9ff2
I20260809 01:45:52.593770 11551 consensus_queue.cc:260] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.594434 10588 master_service.cc:960] ReplaceTablet: received request to replace tablet 012b043d7eef472eb30bdd8e3eec271f from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:52.594710 12780 tablet.cc:1724] T 59de95452c0a4945beae5a9344537e0f P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260809 01:45:52.608933 10580 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet c87361b29664464f9fd73be974e2d3a0 (table default.Table_0708cade7ac74a54aedeb8fe16b3b8df [id=74e764815e2242a9be1ca850a0b6ff30], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet c87361b29664464f9fd73be974e2d3a0 consensus state has no leader
I20260809 01:45:52.608912 12222 ts_tablet_manager.cc:1929] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:52.609308 12222 log.cc:1199] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/60d356fcc48441ca80f2df29b533404f
I20260809 01:45:52.609973 12222 ts_tablet_manager.cc:1950] T 60d356fcc48441ca80f2df29b533404f P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:52.612085 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 275) took 1460 ms. Trace:
W20260809 01:45:52.441426 10804 leader_election.cc:343] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.443707 11553 tablet_bootstrap.cc:492] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:52.629287 11553 ts_tablet_manager.cc:1403] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.268s user 0.008s sys 0.003s
W20260809 01:45:52.629171 10804 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 187763us
W20260809 01:45:52.629926 10804 leader_election.cc:343] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.561394 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 252) took 4489 ms. Trace:
I20260809 01:45:52.630358 10804 leader_election.cc:304] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:52.474902 10805 leader_election.cc:343] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.630266 11951 rpcz_store.cc:276] 0809 01:45:48.072337 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:48.072470 (+ 133us) service_pool.cc:224] Handling call
0809 01:45:52.232854 (+4160384us) ts_tablet_manager.cc:1151] Deleting tablet 72b4850de7d740d794283ca93152fdc1
0809 01:45:52.232898 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.503340 (+270442us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.530420 (+ 27080us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.561334 (+ 30914us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1633,"tablet-delete.queue_time_us":4157196}
W20260809 01:45:52.631021 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 156174us
I20260809 01:45:52.631593 12728 raft_consensus.cc:2749] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:52.631695 10805 leader_election.cc:343] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.632030 10805 leader_election.cc:304] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:52.499929 11949 ts_tablet_manager.cc:1929] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260809 01:45:52.632673 11949 log.cc:1199] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/ead75d766d0346f6823074d90ba8054e
I20260809 01:45:52.633610 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 72b4850de7d740d794283ca93152fdc1 (table default.Table_ff0d273341b54d79b4e634ea694e982f [id=46b096a453084568935bc0d5855cb0f7]) successfully deleted
I20260809 01:45:52.633364 11553 raft_consensus.cc:359] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.634016 11553 raft_consensus.cc:385] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:52.634245 11553 raft_consensus.cc:740] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:52.634820 12727 raft_consensus.cc:2749] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:52.634917 11553 consensus_queue.cc:260] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.507634 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb9afd3be194830ba99f5e20fdaca0a" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:52.633368 11949 ts_tablet_manager.cc:1950] T ead75d766d0346f6823074d90ba8054e P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:52.635905 10910 raft_consensus.cc:2468] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:52.638388 11553 ts_tablet_manager.cc:1434] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260809 01:45:52.456514 12545 leader_election.cc:290] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:52.641367 10804 leader_election.cc:343] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.641772 10804 leader_election.cc:304] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:52.642413 11553 tablet_bootstrap.cc:492] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:52.643151 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 239) took 5439 ms. Trace:
I20260809 01:45:52.643471 11949 rpcz_store.cc:276] 0809 01:45:47.203580 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:47.203716 (+ 136us) service_pool.cc:224] Handling call
0809 01:45:52.017976 (+4814260us) ts_tablet_manager.cc:1151] Deleting tablet ead75d766d0346f6823074d90ba8054e
0809 01:45:52.295483 (+277507us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.349778 (+ 54295us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.499761 (+149983us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.643096 (+143335us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5475,"tablet-delete.queue_time_us":4813703}
I20260809 01:45:52.644649 11949 tablet_replica.cc:333] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:52.654276 11949 raft_consensus.cc:2243] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:52.654819 11949 raft_consensus.cc:2272] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:52.656292 11554 raft_consensus.cc:359] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:52.656953 11554 raft_consensus.cc:385] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:52.657197 11554 raft_consensus.cc:740] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:52.657238 11949 ts_tablet_manager.cc:1916] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:52.662585 11554 consensus_queue.cc:260] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:52.664291 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 250) took 4681 ms. Trace:
I20260809 01:45:52.664805 12222 rpcz_store.cc:276] 0809 01:45:51.151778 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.151942 (+ 164us) service_pool.cc:224] Handling call
0809 01:45:52.032517 (+880575us) ts_tablet_manager.cc:1151] Deleting tablet 60d356fcc48441ca80f2df29b533404f
0809 01:45:52.032577 (+ 60us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.272473 (+239896us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.608791 (+336318us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.612003 (+ 3212us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":879529}
I20260809 01:45:52.666386 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet ead75d766d0346f6823074d90ba8054e (table default.Table_3c356f5b797347c1ab98aaca3d06f4eb [id=028331846d434fc1beff5b11e2129233]) successfully deleted
I20260809 01:45:52.664641 11951 rpcz_store.cc:276] 0809 01:45:47.982746 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:48.069698 (+ 86952us) service_pool.cc:224] Handling call
0809 01:45:52.633573 (+4563875us) ts_tablet_manager.cc:1151] Deleting tablet 750aa437ef594587a512a9c9c8278aa8
0809 01:45:52.657571 (+ 23998us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.664231 (+ 6660us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4557568}
I20260809 01:45:52.668315 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 254) took 4202 ms. Trace:
W20260809 01:45:52.668731 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 750aa437ef594587a512a9c9c8278aa8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 750aa437ef594587a512a9c9c8278aa8 already in progress: creating tablet
I20260809 01:45:52.668620 11951 rpcz_store.cc:276] 0809 01:45:48.466189 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:48.466310 (+ 121us) service_pool.cc:224] Handling call
0809 01:45:52.667888 (+4201578us) ts_tablet_manager.cc:1151] Deleting tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719
0809 01:45:52.667929 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.668257 (+ 328us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4201039}
W20260809 01:45:52.669986 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 already in progress: creating tablet
I20260809 01:45:52.663048 12728 raft_consensus.cc:2749] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:52.664343 10877 tablet_service.cc:1511] Processing CreateTablet for tablet 47e8e02c848d46f48d44b0de414b666d (DEFAULT_TABLE table=default.Table_2b48e5de5f194f2caa53cdd253a8eda4 [id=27956a05763844f8b8b471089c672e20]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:52.670738 10805 consensus_peers.cc:597] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 72b4850de7d740d794283ca93152fdc1. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:52.670704 12545 raft_consensus.cc:493] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:52.637089 11063 leader_election.cc:304] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:52.672035 12545 raft_consensus.cc:515] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.672901 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e8e02c848d46f48d44b0de414b666d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:52.673198 12599 raft_consensus.cc:2804] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:52.673882 12599 raft_consensus.cc:493] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:52.674463 12599 raft_consensus.cc:3060] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:52.674897 12545 leader_election.cc:290] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:52.677170 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f022e6a57a4ec0a5a8c33159e97a8e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:52.615592 11551 ts_tablet_manager.cc:1434] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.091s user 0.001s sys 0.005s
W20260809 01:45:52.683293 10805 leader_election.cc:343] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.685115 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f022e6a57a4ec0a5a8c33159e97a8e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:52.686371 11334 consensus_meta.cc:287] T 21dab4386b434338ad694094a6c036a2 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.535s user 0.009s sys 0.005s
I20260809 01:45:52.686651 11551 tablet_bootstrap.cc:492] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
W20260809 01:45:52.688046 10804 leader_election.cc:343] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.687939 11334 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 277) took 1584 ms. Trace:
I20260809 01:45:52.688571 11554 ts_tablet_manager.cc:1434] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.235s user 0.005s sys 0.002s
I20260809 01:45:52.689868 11951 tablet_replica.cc:333] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
W20260809 01:45:52.691907 10589 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6680c7d6379b43bf8b31e610ddc824d5 consensus state has no leader
I20260809 01:45:52.629151 11351 rpcz_store.cc:276] 0809 01:45:51.092810 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.093431 (+ 621us) service_pool.cc:224] Handling call
0809 01:45:51.197018 (+103587us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:51.197152 (+ 134us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:51.595674 (+398522us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.447724 (+852050us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:52.549607 (+101883us) spinlock_profiling.cc:243] Waited 99.6 ms on lock 0x7b5800000f98. stack: 00007f0ee1f1ecf3 00007f0ee1f1ebf0 00007f0ee1acc574 0000000000516de4 0000000000516d79 00007f0ee1dd9981 00007f0ee1f19521 00007f0ee1f19b52 00007f0eeae871d9 00007f0eea09f4e8 00007f0eea165783 00007f0eea16151f 00007f0eea161cdd 00007f0eea110210 00007f0ee869cbf4 00007f0ee869cb82
0809 01:45:52.549877 (+ 270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229136640}
I20260809 01:45:52.699928 11951 raft_consensus.cc:2243] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:52.700049 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_5aed014baecb416686e9f81b1e185461"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:52.706080 11951 raft_consensus.cc:2272] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:52.707878 11951 ts_tablet_manager.cc:1916] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:52.707746 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_0a2df154a003451eba03bf9545957996"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:52.709095 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a2df154a003451eba03bf9545957996 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:52.710991 12222 tablet_replica.cc:333] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:52.710703 10599 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_001ce3fb9e1e480f915bdbd88c394244" } new_table_name: "default.Table_7399370c91a7495faa66e98683c688d0" modify_external_catalogs: true
I20260809 01:45:52.710732 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 60d356fcc48441ca80f2df29b533404f (table default.Table_29a4bd6bd10a473dbd9dbda008b9eb16 [id=67ee87fabc7749279341528fbcce16d4]) successfully deleted
I20260809 01:45:52.711684 12222 raft_consensus.cc:2243] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:52.712112 12222 raft_consensus.cc:2272] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:52.712495 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_8e736088d5fe4819820a3cc50c3d1e8e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:52.713815 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8e736088d5fe4819820a3cc50c3d1e8e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:52.718266 10597 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 51) took 1067 ms (client timeout 1000 ms). Trace:
W20260809 01:45:52.720906 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5aed014baecb416686e9f81b1e185461 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:52.724257 12222 ts_tablet_manager.cc:1916] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:52.727352 10595 catalog_manager.cc:5719] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66). New cstate: current_term: 1 leader_uuid: "33b5652538e64ed58a09993d3188f8d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:52.728706 10595 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 75e4315b3c644846ac572883c51f23de (table default.Table_ee7c1665ed2344278d1ee031c67438a4 [id=850f362a7f9a4531a3d7d6de29b695ba]). Expected version 4 got 0
I20260809 01:45:52.688462 11334 rpcz_store.cc:276] 0809 01:45:51.315453 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.319587 (+ 4134us) service_pool.cc:224] Handling call
0809 01:45:51.320971 (+ 1384us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:51.321045 (+ 74us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:52.151405 (+830360us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.687137 (+535732us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:52.687859 (+ 722us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:52.729729 10595 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet cef88ffe9e95437da81793795742def3 (table default.Table_c54e4ecef7c145cc970af6169c5a9326 [id=40cb4403a8a94bd3938983441e9254d4]). Expected version 1 got 0
W20260809 01:45:52.718613 10597 rpcz_store.cc:269] 0809 01:45:51.651178 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.651309 (+ 131us) service_pool.cc:224] Handling call
0809 01:45:51.723986 (+ 72677us) spinlock_profiling.cc:243] Waited 70 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:52.718204 (+994218us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:51.903533 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:51.903865 (+ 332us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:51.903885 (+ 20us) write_op.cc:435] Acquired schema lock
0809 01:45:51.903898 (+ 13us) tablet.cc:662] Decoding operations
0809 01:45:51.904218 (+ 320us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:51.904255 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:51.904282 (+ 27us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:51.911041 (+ 6759us) tablet.cc:701] Row locks acquired
0809 01:45:51.911057 (+ 16us) write_op.cc:260] PREPARE: finished
0809 01:45:51.911161 (+ 104us) write_op.cc:270] Start()
0809 01:45:51.911228 (+ 67us) write_op.cc:276] Timestamp: P: 1786239951911089 usec, L: 0
0809 01:45:51.911242 (+ 14us) op_driver.cc:348] REPLICATION: starting
0809 01:45:51.911625 (+ 383us) log.cc:844] Serialized 901 byte log entry
0809 01:45:52.624437 (+712812us) op_driver.cc:464] REPLICATION: finished
0809 01:45:52.643754 (+ 19317us) write_op.cc:301] APPLY: starting
0809 01:45:52.643912 (+ 158us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:52.644155 (+ 243us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:52.644384 (+ 229us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:52.645479 (+ 1095us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:52.645721 (+ 242us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:52.645837 (+ 116us) write_op.cc:312] APPLY: finished
0809 01:45:52.646285 (+ 448us) log.cc:844] Serialized 29 byte log entry
0809 01:45:52.646572 (+ 287us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:52.646665 (+ 93us) write_op.cc:454] Released schema lock
0809 01:45:52.646718 (+ 53us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:52.712277 (+ 65559us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:52.712377 (+ 100us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"spinlock_wait_cycles":161097472,"child_traces":[["op",{"apply.queue_time_us":19082,"apply.run_cpu_time_us":2872,"apply.run_wall_time_us":68536,"num_ops":2,"prepare.queue_time_us":51622,"prepare.run_cpu_time_us":1751,"prepare.run_wall_time_us":8247,"replication_time_us":713055}]]}
I20260809 01:45:52.750186 10595 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]). Expected version 3 got 0
I20260809 01:45:52.770224 11551 tablet_bootstrap.cc:654] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:52.779392 12599 raft_consensus.cc:515] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.722357 12783 tablet.cc:1724] T 59de95452c0a4945beae5a9344537e0f P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:52.784294 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb9afd3be194830ba99f5e20fdaca0a" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:52.737738 12545 raft_consensus.cc:493] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:52.785668 12545 raft_consensus.cc:515] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:52.786408 11062 leader_election.cc:343] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.740417 11553 tablet_bootstrap.cc:654] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:52.766430 11949 ts_tablet_manager.cc:1929] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:52.787359 11949 log.cc:1199] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/977694496e014ef29f5d7d513e5219c0
I20260809 01:45:52.788276 11949 ts_tablet_manager.cc:1950] T 977694496e014ef29f5d7d513e5219c0 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:52.789911 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8baa4ae62edf46b5a386661a6826bbab" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:52.791002 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 250) took 5096 ms. Trace:
W20260809 01:45:52.791770 10805 leader_election.cc:343] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.792686 12545 leader_election.cc:290] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:52.793437 10804 leader_election.cc:304] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:52.794159 10804 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 106156us
I20260809 01:45:52.791345 11949 rpcz_store.cc:276] 0809 01:45:47.694387 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:47.694930 (+ 543us) service_pool.cc:224] Handling call
0809 01:45:52.644471 (+4949541us) ts_tablet_manager.cc:1151] Deleting tablet 977694496e014ef29f5d7d513e5219c0
0809 01:45:52.644528 (+ 57us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.726765 (+ 82237us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.766214 (+ 39449us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.790871 (+ 24657us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4948825}
I20260809 01:45:52.798327 11949 tablet_replica.cc:333] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:52.798355 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8baa4ae62edf46b5a386661a6826bbab" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:52.799019 11949 raft_consensus.cc:2243] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:52.799461 11949 raft_consensus.cc:2272] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:52.800414 12545 raft_consensus.cc:2749] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:52.801062 11949 ts_tablet_manager.cc:1916] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:52.801422 11554 tablet_bootstrap.cc:492] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:52.804116 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 977694496e014ef29f5d7d513e5219c0 (table default.Table_0908ea7070134c749de4ff154f23628e [id=148da762500547349a600027071299ab]) successfully deleted
W20260809 01:45:52.807410 10804 leader_election.cc:343] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.807772 10804 leader_election.cc:304] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:52.810747 12545 raft_consensus.cc:2749] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:52.816077 11951 ts_tablet_manager.cc:1929] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:52.832581 11554 tablet_bootstrap.cc:654] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:52.833302 11951 log.cc:1199] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/9b51f999fab24b1ca7451ff543a124a4
I20260809 01:45:52.835862 11949 ts_tablet_manager.cc:1929] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:52.836344 11949 log.cc:1199] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/19fe81e277cd4c798a07ceafa0b60566
I20260809 01:45:52.837158 11951 ts_tablet_manager.cc:1950] T 9b51f999fab24b1ca7451ff543a124a4 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:52.837198 11949 ts_tablet_manager.cc:1950] T 19fe81e277cd4c798a07ceafa0b60566 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:52.846714 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 255) took 4345 ms. Trace:
I20260809 01:45:52.857847 10597 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_0908ea7070134c749de4ff154f23628e" } modify_external_catalogs: true
I20260809 01:45:52.858551 10597 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_0908ea7070134c749de4ff154f23628e" } modify_external_catalogs: true
I20260809 01:45:52.883113 11554 tablet_bootstrap.cc:492] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:52.883585 11554 ts_tablet_manager.cc:1403] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.194s user 0.006s sys 0.013s
I20260809 01:45:52.872565 11951 rpcz_store.cc:276] 0809 01:45:48.501131 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:48.501252 (+ 121us) service_pool.cc:224] Handling call
0809 01:45:52.685509 (+4184257us) ts_tablet_manager.cc:1151] Deleting tablet 9b51f999fab24b1ca7451ff543a124a4
0809 01:45:52.689756 (+ 4247us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.754862 (+ 65106us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.815956 (+ 61094us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.846655 (+ 30699us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4167635}
I20260809 01:45:52.886736 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 9b51f999fab24b1ca7451ff543a124a4 (table default.Table_5264c640dfc04d9bae010805cdaf4ea6 [id=c7b50f12c7934944aceb691a5d7b0609]) successfully deleted
I20260809 01:45:52.887913 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 256) took 4262 ms. Trace:
I20260809 01:45:52.888384 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 251) took 5118 ms. Trace:
I20260809 01:45:52.888710 11949 rpcz_store.cc:276] 0809 01:45:47.770476 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:47.770584 (+ 108us) service_pool.cc:224] Handling call
0809 01:45:52.798171 (+5027587us) ts_tablet_manager.cc:1151] Deleting tablet 19fe81e277cd4c798a07ceafa0b60566
0809 01:45:52.798227 (+ 56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.823776 (+ 25549us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.835702 (+ 11926us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.888317 (+ 52615us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5026802}
I20260809 01:45:52.890030 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 19fe81e277cd4c798a07ceafa0b60566 (table default.Table_1998ec81597a4c4584dc4e4f90caa274 [id=cb347b72c1cf436095c6043c5b169641]) successfully deleted
I20260809 01:45:52.890699 12545 raft_consensus.cc:493] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:52.891175 12545 raft_consensus.cc:515] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:52.892403 12599 leader_election.cc:290] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:52.892628 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb9afd3be194830ba99f5e20fdaca0a" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:52.893064 12545 leader_election.cc:290] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:52.893107 10910 raft_consensus.cc:3060] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:52.893688 11951 rpcz_store.cc:276] 0809 01:45:48.625908 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:48.626020 (+ 112us) service_pool.cc:224] Handling call
0809 01:45:52.887669 (+4261649us) ts_tablet_manager.cc:1151] Deleting tablet 12fb852b8ea8478cb160a265459865c4
0809 01:45:52.887707 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.887861 (+ 154us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4260987}
I20260809 01:45:52.895560 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38bd344108649ed8e1687f82154000b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:52.897676 10804 leader_election.cc:343] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.898795 12792 raft_consensus.cc:493] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 629f002048f64491afdd57f72a466e59)
I20260809 01:45:52.899592 10910 raft_consensus.cc:2468] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:52.899806 11553 tablet_bootstrap.cc:492] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:52.899536 12792 raft_consensus.cc:515] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:52.900207 11553 ts_tablet_manager.cc:1403] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.261s user 0.013s sys 0.006s
I20260809 01:45:52.900532 11063 leader_election.cc:304] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:52.901567 12599 raft_consensus.cc:2804] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:52.902670 12599 raft_consensus.cc:697] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:52.902662 11553 raft_consensus.cc:359] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:52.903223 11553 raft_consensus.cc:385] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:52.903482 11553 raft_consensus.cc:740] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:52.903460 12599 consensus_queue.cc:237] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.904109 11553 consensus_queue.cc:260] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:52.907420 11551 tablet_bootstrap.cc:492] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:52.907840 11551 ts_tablet_manager.cc:1403] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.221s user 0.009s sys 0.008s
I20260809 01:45:52.908334 11553 ts_tablet_manager.cc:1434] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260809 01:45:52.896723 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38bd344108649ed8e1687f82154000b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:52.909313 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 12fb852b8ea8478cb160a265459865c4 because the tablet was not found. No further retry: Not found: Tablet not found: 12fb852b8ea8478cb160a265459865c4
W20260809 01:45:52.909648 10805 leader_election.cc:343] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.910032 10805 leader_election.cc:304] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:52.910566 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 257) took 4060 ms. Trace:
I20260809 01:45:52.911134 12545 raft_consensus.cc:2749] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:52.910928 11951 rpcz_store.cc:276] 0809 01:45:48.850137 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:48.851672 (+ 1535us) service_pool.cc:224] Handling call
0809 01:45:52.910184 (+4058512us) ts_tablet_manager.cc:1151] Deleting tablet d146bdaf4dd2418ca870c4afeb6ccb6c
0809 01:45:52.910237 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.910513 (+ 276us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4057692}
I20260809 01:45:52.888247 11554 raft_consensus.cc:359] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.912318 11554 raft_consensus.cc:385] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 01:45:52.912400 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet d146bdaf4dd2418ca870c4afeb6ccb6c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d146bdaf4dd2418ca870c4afeb6ccb6c already in progress: creating tablet
I20260809 01:45:52.912575 11554 raft_consensus.cc:740] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:52.913504 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 258) took 4007 ms. Trace:
I20260809 01:45:52.914103 11553 tablet_bootstrap.cc:492] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:52.913872 11951 rpcz_store.cc:276] 0809 01:45:48.906361 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:48.906480 (+ 119us) service_pool.cc:224] Handling call
0809 01:45:52.913155 (+4006675us) ts_tablet_manager.cc:1151] Deleting tablet 81da8d08131d4a3ba636ecfc7a7d2951
0809 01:45:52.913206 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.913462 (+ 256us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4006137}
I20260809 01:45:52.914602 11949 tablet_replica.cc:333] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
W20260809 01:45:52.915174 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 81da8d08131d4a3ba636ecfc7a7d2951 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81da8d08131d4a3ba636ecfc7a7d2951 already in progress: creating tablet
I20260809 01:45:52.916177 11949 raft_consensus.cc:2243] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:52.916617 11949 raft_consensus.cc:2272] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:52.918277 11949 ts_tablet_manager.cc:1916] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:52.916170 11554 consensus_queue.cc:260] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.925625 11551 raft_consensus.cc:359] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.926218 11551 raft_consensus.cc:385] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:52.926501 11551 raft_consensus.cc:740] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:52.934523 12728 raft_consensus.cc:493] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:52.934981 12728 raft_consensus.cc:515] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.946062 12792 leader_election.cc:290] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:52.946875 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa12353c25cb4da680d478ce104bc6ba" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:52.947535 12728 leader_election.cc:290] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:52.947959 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09a73ee8e8ec496a984efcab27715506" candidate_term: 3 candidate_status { last_received { term: 2 index: 421 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:45:52.948663 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa12353c25cb4da680d478ce104bc6ba" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:52.949497 10804 leader_election.cc:343] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.916267 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 259) took 3836 ms. Trace:
W20260809 01:45:52.950510 10805 leader_election.cc:343] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:52.950887 10805 leader_election.cc:304] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:52.949958 11951 rpcz_store.cc:276] 0809 01:45:49.079576 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:49.079684 (+ 108us) service_pool.cc:224] Handling call
0809 01:45:52.915982 (+3836298us) ts_tablet_manager.cc:1151] Deleting tablet bb7a8b6d06064addb02979a7feedf756
0809 01:45:52.916016 (+ 34us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.916221 (+ 205us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3835768}
W20260809 01:45:52.952524 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet bb7a8b6d06064addb02979a7feedf756 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb7a8b6d06064addb02979a7feedf756 already in progress: creating tablet
I20260809 01:45:52.957832 11551 consensus_queue.cc:260] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:52.960258 12728 raft_consensus.cc:2749] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:52.962616 11551 ts_tablet_manager.cc:1434] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.045s user 0.006s sys 0.000s
I20260809 01:45:52.962806 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 260) took 2964 ms. Trace:
I20260809 01:45:52.963174 11951 rpcz_store.cc:276] 0809 01:45:49.998290 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:49.998425 (+ 135us) service_pool.cc:224] Handling call
0809 01:45:52.962372 (+2963947us) ts_tablet_manager.cc:1151] Deleting tablet 117d5fad26fd42dab19cfc1687364506
0809 01:45:52.962436 (+ 64us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.962737 (+ 301us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2963357}
W20260809 01:45:52.964768 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 117d5fad26fd42dab19cfc1687364506 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 117d5fad26fd42dab19cfc1687364506 already in progress: creating tablet
I20260809 01:45:52.971279 11949 ts_tablet_manager.cc:1929] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:52.975879 12222 ts_tablet_manager.cc:1929] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:52.976300 12222 log.cc:1199] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/36a542c773b7438ead16c5b3af224cf9
I20260809 01:45:52.976938 12222 ts_tablet_manager.cc:1950] T 36a542c773b7438ead16c5b3af224cf9 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
W20260809 01:45:52.979226 10320 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
I20260809 01:45:52.980139 11949 log.cc:1199] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/e93334cc766044e79d37f293bc8556fb
I20260809 01:45:52.981062 11949 ts_tablet_manager.cc:1950] T e93334cc766044e79d37f293bc8556fb P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:52.986956 10592 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet b9493892c1ff48e7a0558ff0fcb7a0f3
I20260809 01:45:52.988731 10592 catalog_manager.cc:6807] ReplaceTablet: tablet b9493892c1ff48e7a0558ff0fcb7a0f3 deleted and replaced by tablet 83dd3b3eb484407bbef0e54581b759dd
I20260809 01:45:52.988909 10588 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 012b043d7eef472eb30bdd8e3eec271f
I20260809 01:45:52.990458 10588 catalog_manager.cc:6807] ReplaceTablet: tablet 012b043d7eef472eb30bdd8e3eec271f deleted and replaced by tablet db32b031d5a8423a8196a5595f24145c
W20260809 01:45:52.991780 10599 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2126670a2f29408a9ba4b949f966bf76 has no consensus state
I20260809 01:45:52.992080 10877 tablet_service.cc:1558] Processing DeleteTablet for tablet b9493892c1ff48e7a0558ff0fcb7a0f3 with delete_type TABLET_DATA_DELETED (replaced by tablet 83dd3b3eb484407bbef0e54581b759dd) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:52.992733 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet 012b043d7eef472eb30bdd8e3eec271f with delete_type TABLET_DATA_DELETED (replaced by tablet db32b031d5a8423a8196a5595f24145c) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:52.993419 11551 tablet_bootstrap.cc:492] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:52.995021 11354 tablet_service.cc:1558] Processing DeleteTablet for tablet b9493892c1ff48e7a0558ff0fcb7a0f3 with delete_type TABLET_DATA_DELETED (replaced by tablet 83dd3b3eb484407bbef0e54581b759dd) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:52.995801 11334 tablet_service.cc:1558] Processing DeleteTablet for tablet 012b043d7eef472eb30bdd8e3eec271f with delete_type TABLET_DATA_DELETED (replaced by tablet db32b031d5a8423a8196a5595f24145c) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:52.999609 11551 tablet_bootstrap.cc:654] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:53.002066 10591 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]). Expected version 3 got 0
I20260809 01:45:53.002718 10591 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet cef88ffe9e95437da81793795742def3 (table default.Table_c54e4ecef7c145cc970af6169c5a9326 [id=40cb4403a8a94bd3938983441e9254d4]). Expected version 1 got 0
I20260809 01:45:53.003438 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 252) took 5190 ms. Trace:
I20260809 01:45:53.003741 11949 rpcz_store.cc:276] 0809 01:45:47.812961 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:47.813078 (+ 117us) service_pool.cc:224] Handling call
0809 01:45:52.906341 (+5093263us) ts_tablet_manager.cc:1151] Deleting tablet e93334cc766044e79d37f293bc8556fb
0809 01:45:52.907373 (+ 1032us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.964199 (+ 56826us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.971167 (+ 6968us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.003379 (+ 32212us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5092448}
I20260809 01:45:53.004988 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet e93334cc766044e79d37f293bc8556fb (table default.Table_2b01d8bf86d44f96b64752f511b02f50 [id=7cede7d9f9bf44f997b90481537918dc]) successfully deleted
I20260809 01:45:53.006156 10588 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_25a94901452a4afba66db1eedb447239"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:53.006840 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 274) took 1855 ms. Trace:
W20260809 01:45:53.007951 10588 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25a94901452a4afba66db1eedb447239 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:53.010274 10589 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]). Expected version 3 got 0
I20260809 01:45:53.011869 10589 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet cef88ffe9e95437da81793795742def3 (table default.Table_c54e4ecef7c145cc970af6169c5a9326 [id=40cb4403a8a94bd3938983441e9254d4]). Expected version 1 got 0
I20260809 01:45:53.009196 10592 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_78c91fd32f104ad785b008e345a24e04" } modify_external_catalogs: true
I20260809 01:45:53.014169 10592 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_78c91fd32f104ad785b008e345a24e04" } modify_external_catalogs: true
I20260809 01:45:53.015108 10589 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet c87361b29664464f9fd73be974e2d3a0 (table default.Table_0708cade7ac74a54aedeb8fe16b3b8df [id=74e764815e2242a9be1ca850a0b6ff30]). Expected version 1 got 0
I20260809 01:45:53.013273 10599 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_6e83741f1d33450c91d06322e7b2ef98" } new_table_name: "default.Table_bd6eedffdf6440efa3e776b7079015d8" modify_external_catalogs: true
I20260809 01:45:53.007150 12222 rpcz_store.cc:276] 0809 01:45:51.150984 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.166152 (+ 15168us) service_pool.cc:224] Handling call
0809 01:45:52.710846 (+1544694us) ts_tablet_manager.cc:1151] Deleting tablet 36a542c773b7438ead16c5b3af224cf9
0809 01:45:52.710886 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:52.844778 (+133892us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:52.975730 (+130952us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.006781 (+ 31051us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1544137}
I20260809 01:45:53.008111 12599 raft_consensus.cc:493] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:53.017132 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 279) took 1552 ms. Trace:
I20260809 01:45:53.017273 12599 raft_consensus.cc:515] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.017992 11113 tablet_service.cc:1558] Processing DeleteTablet for tablet 012b043d7eef472eb30bdd8e3eec271f with delete_type TABLET_DATA_DELETED (replaced by tablet db32b031d5a8423a8196a5595f24145c) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:53.019255 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 36a542c773b7438ead16c5b3af224cf9 (table default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]) successfully deleted
I20260809 01:45:53.019903 11110 tablet_service.cc:1558] Processing DeleteTablet for tablet b9493892c1ff48e7a0558ff0fcb7a0f3 with delete_type TABLET_DATA_DELETED (replaced by tablet 83dd3b3eb484407bbef0e54581b759dd) from {username='slave'} at 127.0.0.1:56566
W20260809 01:45:53.022336 11063 leader_election.cc:343] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 995a37a9e6b14889b876da5d76c89b84
I20260809 01:45:53.022857 12599 leader_election.cc:290] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:53.017501 12222 rpcz_store.cc:276] 0809 01:45:51.464628 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.464829 (+ 201us) service_pool.cc:224] Handling call
0809 01:45:53.016743 (+1551914us) ts_tablet_manager.cc:1151] Deleting tablet 991e8580b8624f34881d56c6cc60600f
0809 01:45:53.016797 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.017081 (+ 284us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1486873}
I20260809 01:45:53.030120 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 262) took 2999 ms. Trace:
I20260809 01:45:53.030334 11949 tablet_replica.cc:333] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:53.030385 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 281) took 1406 ms. Trace:
I20260809 01:45:53.030462 11951 rpcz_store.cc:276] 0809 01:45:50.030414 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:50.030550 (+ 136us) service_pool.cc:224] Handling call
0809 01:45:53.029730 (+2999180us) ts_tablet_manager.cc:1151] Deleting tablet 14cd02276f26405c95f028edf967bf10
0809 01:45:53.029779 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.030057 (+ 278us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2998639}
I20260809 01:45:53.030668 12222 rpcz_store.cc:276] 0809 01:45:51.623807 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.623947 (+ 140us) service_pool.cc:224] Handling call
0809 01:45:53.030055 (+1406108us) ts_tablet_manager.cc:1151] Deleting tablet 7634fe763a4e4fec91c5938e45a5d1fc
0809 01:45:53.030096 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.030331 (+ 235us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1405580}
I20260809 01:45:53.031672 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 266) took 2753 ms. Trace:
W20260809 01:45:53.031795 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 14cd02276f26405c95f028edf967bf10 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14cd02276f26405c95f028edf967bf10 already in progress: creating tablet
I20260809 01:45:53.031948 11951 rpcz_store.cc:276] 0809 01:45:50.278821 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:50.285272 (+ 6451us) service_pool.cc:224] Handling call
0809 01:45:53.031340 (+2746068us) ts_tablet_manager.cc:1151] Deleting tablet a38bd344108649ed8e1687f82154000b
0809 01:45:53.031378 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.031614 (+ 236us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2745400}
I20260809 01:45:53.033075 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 267) took 2753 ms. Trace:
W20260809 01:45:53.033223 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet a38bd344108649ed8e1687f82154000b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a38bd344108649ed8e1687f82154000b already in progress: creating tablet
I20260809 01:45:53.034132 11949 raft_consensus.cc:2243] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:53.031584 12222 tablet_replica.cc:333] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:53.035792 12222 raft_consensus.cc:2243] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:53.036228 11949 raft_consensus.cc:2272] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:53.036231 12222 raft_consensus.cc:2272] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:53.037851 12222 ts_tablet_manager.cc:1916] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:53.038038 11949 ts_tablet_manager.cc:1916] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:53.042775 10598 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 75e4315b3c644846ac572883c51f23de (table default.Table_ee7c1665ed2344278d1ee031c67438a4 [id=850f362a7f9a4531a3d7d6de29b695ba]). Expected version 4 got 0
W20260809 01:45:53.044216 10320 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.125:42063, user_credentials={real_user=slave}} blocked reactor thread for 65043.2us
W20260809 01:45:53.053128 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 991e8580b8624f34881d56c6cc60600f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 991e8580b8624f34881d56c6cc60600f already in progress: creating tablet
W20260809 01:45:53.055342 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 7634fe763a4e4fec91c5938e45a5d1fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7634fe763a4e4fec91c5938e45a5d1fc already in progress: creating tablet
I20260809 01:45:53.043737 10598 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet cef88ffe9e95437da81793795742def3 (table default.Table_c54e4ecef7c145cc970af6169c5a9326 [id=40cb4403a8a94bd3938983441e9254d4]). Expected version 1 got 0
I20260809 01:45:53.033375 11951 rpcz_store.cc:276] 0809 01:45:50.279253 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:50.286030 (+ 6777us) service_pool.cc:224] Handling call
0809 01:45:53.032769 (+2746739us) ts_tablet_manager.cc:1151] Deleting tablet ebc209c696ac4c669961f5ed8392af40
0809 01:45:53.032811 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.033028 (+ 217us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2746198}
W20260809 01:45:53.066944 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet ebc209c696ac4c669961f5ed8392af40 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ebc209c696ac4c669961f5ed8392af40 already in progress: creating tablet
I20260809 01:45:53.065693 10598 catalog_manager.cc:5719] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66). New cstate: current_term: 1 leader_uuid: "33b5652538e64ed58a09993d3188f8d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:53.070489 10598 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet cfb9afd3be194830ba99f5e20fdaca0a (table default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]). Expected version 1 got 0
I20260809 01:45:53.068018 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 276) took 1965 ms. Trace:
I20260809 01:45:53.077857 11951 rpcz_store.cc:276] 0809 01:45:51.102160 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.102297 (+ 137us) service_pool.cc:224] Handling call
0809 01:45:53.067671 (+1965374us) ts_tablet_manager.cc:1151] Deleting tablet 1c73bed7a9774cbc8ac774e6514a981d
0809 01:45:53.067714 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.067962 (+ 248us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1941826}
I20260809 01:45:53.083715 11554 ts_tablet_manager.cc:1434] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.200s user 0.005s sys 0.003s
I20260809 01:45:53.088716 11384 raft_consensus.cc:3060] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:53.089102 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "995a37a9e6b14889b876da5d76c89b84" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:53.095486 11554 tablet_bootstrap.cc:492] T 83ac8498dff04b4e93303d7a35e62134 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
W20260809 01:45:53.095625 11062 leader_election.cc:343] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:53.096046 11062 leader_election.cc:304] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:53.079802 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 1c73bed7a9774cbc8ac774e6514a981d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c73bed7a9774cbc8ac774e6514a981d already in progress: creating tablet
I20260809 01:45:53.099529 12599 raft_consensus.cc:2749] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:53.083678 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet b095338eb59e4161aaaf3afb762cd296
I20260809 01:45:53.100531 10598 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]). Expected version 3 got 0
I20260809 01:45:53.103214 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet b095338eb59e4161aaaf3afb762cd296 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:52 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:53.103361 11951 tablet_replica.cc:333] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:53.104103 11949 ts_tablet_manager.cc:1929] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20260809 01:45:53.104445 11949 log.cc:1199] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/72b4850de7d740d794283ca93152fdc1
I20260809 01:45:53.105139 11949 ts_tablet_manager.cc:1950] T 72b4850de7d740d794283ca93152fdc1 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:53.106700 11334 tablet_service.cc:1558] Processing DeleteTablet for tablet b095338eb59e4161aaaf3afb762cd296 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:52 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:53.109911 11553 tablet_bootstrap.cc:654] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:53.104182 11951 raft_consensus.cc:2243] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:53.112284 11951 raft_consensus.cc:2272] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:53.114010 11951 ts_tablet_manager.cc:1916] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:53.102877 11110 tablet_service.cc:1558] Processing DeleteTablet for tablet b095338eb59e4161aaaf3afb762cd296 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:52 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:53.124642 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 256) took 5051 ms. Trace:
I20260809 01:45:53.127136 11551 tablet_bootstrap.cc:492] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:53.128007 11551 ts_tablet_manager.cc:1403] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.135s user 0.006s sys 0.005s
W20260809 01:45:53.130918 10320 leader_election.cc:336] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063): Network error: Client connection negotiation failed: client connection to 127.10.8.125:42063: connect: Connection refused (error 111)
I20260809 01:45:53.131589 10320 leader_election.cc:304] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09a73ee8e8ec496a984efcab27715506; no voters: 629f002048f64491afdd57f72a466e59, 71bc72deb1d2402cabd817dfe3b27e88
I20260809 01:45:53.136584 12792 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 01:45:53.137584 11551 raft_consensus.cc:359] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:53.138312 11551 raft_consensus.cc:385] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:53.137058 11949 rpcz_store.cc:276] 0809 01:45:48.073575 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:48.100271 (+ 26696us) service_pool.cc:224] Handling call
0809 01:45:53.030055 (+4929784us) ts_tablet_manager.cc:1151] Deleting tablet 72b4850de7d740d794283ca93152fdc1
0809 01:45:53.030109 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.053654 (+ 23545us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.103973 (+ 50319us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.124579 (+ 20606us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4927405}
I20260809 01:45:53.139070 11551 raft_consensus.cc:740] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:53.140283 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 72b4850de7d740d794283ca93152fdc1 (table default.Table_ff0d273341b54d79b4e634ea694e982f [id=46b096a453084568935bc0d5855cb0f7]) successfully deleted
I20260809 01:45:53.141794 11551 consensus_queue.cc:260] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:53.143062 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 254) took 5070 ms. Trace:
I20260809 01:45:53.147142 11949 rpcz_store.cc:276] 0809 01:45:48.072409 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:48.084171 (+ 11762us) service_pool.cc:224] Handling call
0809 01:45:53.141296 (+5057125us) ts_tablet_manager.cc:1151] Deleting tablet 750aa437ef594587a512a9c9c8278aa8
0809 01:45:53.141391 (+ 95us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.142988 (+ 1597us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5027819}
I20260809 01:45:53.151480 11554 tablet_bootstrap.cc:654] T 83ac8498dff04b4e93303d7a35e62134 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:53.152416 11551 ts_tablet_manager.cc:1434] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.024s user 0.007s sys 0.002s
I20260809 01:45:53.153465 11551 tablet_bootstrap.cc:492] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:53.163697 11551 tablet_bootstrap.cc:654] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:53.164841 10599 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 63c1a610663a4a2fa58f6cdc5f8d86ce consensus state has no leader
I20260809 01:45:53.179514 12653 raft_consensus.cc:493] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:53.180152 12653 raft_consensus.cc:515] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:53.188234 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 750aa437ef594587a512a9c9c8278aa8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 750aa437ef594587a512a9c9c8278aa8 already in progress: creating tablet
I20260809 01:45:53.192389 12599 consensus_queue.cc:1048] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:53.192137 11949 tablet_replica.cc:333] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:53.208119 10599 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_59150b4860b64269a5f1712f8b8f0d30" } new_table_name: "default.Table_99bad24613e94b6f840f637bea4978fc" modify_external_catalogs: true
I20260809 01:45:53.210498 11949 raft_consensus.cc:2243] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:53.211007 11949 raft_consensus.cc:2272] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:53.210690 10588 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_d21ec0ae97254d549278ab5c7e3e6389"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:53.210907 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_51cdbe440ce64783a1ead0a751db0e3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:53.212349 10588 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d21ec0ae97254d549278ab5c7e3e6389 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:53.212713 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51cdbe440ce64783a1ead0a751db0e3e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:53.214231 12222 ts_tablet_manager.cc:1929] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:53.214624 12222 log.cc:1199] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/b84f1943c1a34e999fc7c12a5847cdb6
I20260809 01:45:53.215224 12222 ts_tablet_manager.cc:1950] T b84f1943c1a34e999fc7c12a5847cdb6 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:53.217135 12222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 283) took 1375 ms. Trace:
I20260809 01:45:53.219561 12778 raft_consensus.cc:493] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:53.220149 12778 raft_consensus.cc:515] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:53.221944 12728 raft_consensus.cc:493] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:53.222419 12728 raft_consensus.cc:515] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:53.224864 12728 leader_election.cc:290] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:53.226676 11951 ts_tablet_manager.cc:1929] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:53.227058 11951 log.cc:1199] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/5050a590fe0240ac8c9f9191ec2475a1
I20260809 01:45:53.227370 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad9cdaa5c0a4dc7bfc34db8f18c2b6b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:53.227842 11951 ts_tablet_manager.cc:1950] T 5050a590fe0240ac8c9f9191ec2475a1 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:53.217406 12222 rpcz_store.cc:276] 0809 01:45:51.842087 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.842227 (+ 140us) service_pool.cc:224] Handling call
0809 01:45:53.031449 (+1189222us) ts_tablet_manager.cc:1151] Deleting tablet b84f1943c1a34e999fc7c12a5847cdb6
0809 01:45:53.031486 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.138105 (+106619us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.214065 (+ 75960us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.217081 (+ 3016us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1094152}
W20260809 01:45:53.229211 10804 leader_election.cc:343] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:53.229259 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_840104257b8e416281e21b88d91837cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:53.229962 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet b84f1943c1a34e999fc7c12a5847cdb6 (table default.Table_e39314f303ee429f897d4f85b8edbf69 [id=21bc1ee96b724341904d6b0c5b10fb7a]) successfully deleted
W20260809 01:45:53.230974 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_840104257b8e416281e21b88d91837cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:53.232132 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad9cdaa5c0a4dc7bfc34db8f18c2b6b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:53.234325 10805 leader_election.cc:343] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:53.235064 10805 leader_election.cc:304] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:53.236315 12728 raft_consensus.cc:2749] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:53.238293 12653 leader_election.cc:290] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
W20260809 01:45:53.239701 11289 leader_election.cc:343] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 60d356fcc48441ca80f2df29b533404f
W20260809 01:45:53.240341 11283 leader_election.cc:343] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: 60d356fcc48441ca80f2df29b533404f
I20260809 01:45:53.240731 11283 leader_election.cc:304] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:53.241505 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 012b043d7eef472eb30bdd8e3eec271f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 012b043d7eef472eb30bdd8e3eec271f already in progress: creating tablet
I20260809 01:45:53.242347 12653 raft_consensus.cc:2749] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:53.240327 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 279) took 1924 ms. Trace:
I20260809 01:45:53.243188 11951 rpcz_store.cc:276] 0809 01:45:51.316242 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.316502 (+ 260us) service_pool.cc:224] Handling call
0809 01:45:53.103202 (+1786700us) ts_tablet_manager.cc:1151] Deleting tablet 5050a590fe0240ac8c9f9191ec2475a1
0809 01:45:53.103244 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.190376 (+ 87132us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.226552 (+ 36176us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.240269 (+ 13717us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1785959}
I20260809 01:45:53.244318 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 5050a590fe0240ac8c9f9191ec2475a1 (table default.Table_fe1acd943c5c42c9b09a0a5845cbee68 [id=487f10bf79a944168eece2b1e8a2daf0]) successfully deleted
I20260809 01:45:53.244903 12778 leader_election.cc:290] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:53.246162 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c528b45de0d14c19af5d6836b7a14f1c" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:53.247401 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet b9493892c1ff48e7a0558ff0fcb7a0f3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9493892c1ff48e7a0558ff0fcb7a0f3 already in progress: creating tablet
W20260809 01:45:53.250083 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet b095338eb59e4161aaaf3afb762cd296 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b095338eb59e4161aaaf3afb762cd296 already in progress: creating tablet
I20260809 01:45:53.250756 11951 tablet_replica.cc:333] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:53.251446 11951 raft_consensus.cc:2243] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:53.251900 11951 raft_consensus.cc:2272] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:53.256083 10592 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet db32b031d5a8423a8196a5595f24145c
I20260809 01:45:53.213320 11949 ts_tablet_manager.cc:1916] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:53.248029 11062 leader_election.cc:343] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:53.262040 11951 ts_tablet_manager.cc:1916] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:53.272923 11554 tablet_bootstrap.cc:492] T 83ac8498dff04b4e93303d7a35e62134 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:53.273311 11554 ts_tablet_manager.cc:1403] T 83ac8498dff04b4e93303d7a35e62134 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.178s user 0.004s sys 0.008s
W20260809 01:45:53.273250 11063 leader_election.cc:343] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: c528b45de0d14c19af5d6836b7a14f1c
I20260809 01:45:53.273689 11063 leader_election.cc:304] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:53.275081 12778 raft_consensus.cc:2749] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:53.275751 11554 raft_consensus.cc:359] T 83ac8498dff04b4e93303d7a35e62134 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.276371 11554 raft_consensus.cc:385] T 83ac8498dff04b4e93303d7a35e62134 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:53.276623 11554 raft_consensus.cc:740] T 83ac8498dff04b4e93303d7a35e62134 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:53.277308 11554 consensus_queue.cc:260] T 83ac8498dff04b4e93303d7a35e62134 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.283123 11951 ts_tablet_manager.cc:1929] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:53.283592 11951 log.cc:1199] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/c16897857df24c6da5744a99d5f687c0
I20260809 01:45:53.284412 11951 ts_tablet_manager.cc:1950] T c16897857df24c6da5744a99d5f687c0 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:53.298727 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 280) took 1981 ms. Trace:
I20260809 01:45:53.299095 11951 rpcz_store.cc:276] 0809 01:45:51.317671 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.317765 (+ 94us) service_pool.cc:224] Handling call
0809 01:45:53.250603 (+1932838us) ts_tablet_manager.cc:1151] Deleting tablet c16897857df24c6da5744a99d5f687c0
0809 01:45:53.250649 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.269411 (+ 18762us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.282988 (+ 13577us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.298661 (+ 15673us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1932161}
I20260809 01:45:53.300491 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet c16897857df24c6da5744a99d5f687c0 (table default.Table_6d4e0b3afc864450bdf1a8fb35f7dc75 [id=9b307bc98a124b07a2f46fafb20ce1f0]) successfully deleted
I20260809 01:45:53.301582 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 281) took 1982 ms. Trace:
I20260809 01:45:53.301927 11951 rpcz_store.cc:276] 0809 01:45:51.362467 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.362627 (+ 160us) service_pool.cc:224] Handling call
0809 01:45:53.301271 (+1938644us) ts_tablet_manager.cc:1151] Deleting tablet 5af35cf31a66474cb0575b2fb1547616
0809 01:45:53.301314 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.301503 (+ 189us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1937939}
W20260809 01:45:53.303102 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 5af35cf31a66474cb0575b2fb1547616 because the tablet was not found. No further retry: Not found: Tablet not found: 5af35cf31a66474cb0575b2fb1547616
I20260809 01:45:53.305583 10592 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_d917f500390043af96190d13685091ca" } modify_external_catalogs: true
I20260809 01:45:53.306077 10592 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_d917f500390043af96190d13685091ca" } modify_external_catalogs: true
I20260809 01:45:53.306329 11554 ts_tablet_manager.cc:1434] T 83ac8498dff04b4e93303d7a35e62134 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.033s user 0.005s sys 0.000s
I20260809 01:45:53.312726 11554 tablet_bootstrap.cc:492] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:53.315354 11553 tablet_bootstrap.cc:492] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:53.315794 11553 ts_tablet_manager.cc:1403] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.402s user 0.008s sys 0.017s
I20260809 01:45:53.316571 11951 tablet_replica.cc:333] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:53.317359 11951 raft_consensus.cc:2243] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:53.318421 11951 pending_rounds.cc:70] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316: Trying to abort 1 pending ops.
I20260809 01:45:53.337955 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_6f67b7622b1640ec9bc317480a93535b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:53.339819 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f67b7622b1640ec9bc317480a93535b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:53.344074 11951 pending_rounds.cc:77] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438840683167744 op_type: NO_OP noop_request { }
I20260809 01:45:53.321408 11554 tablet_bootstrap.cc:654] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:53.345355 11951 raft_consensus.cc:2889] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:45:53.345793 11553 raft_consensus.cc:359] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.346503 11553 raft_consensus.cc:385] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:53.346796 11553 raft_consensus.cc:740] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:53.346640 11951 raft_consensus.cc:2272] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:53.347458 11553 consensus_queue.cc:260] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.350281 11951 ts_tablet_manager.cc:1916] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:53.364459 11553 ts_tablet_manager.cc:1434] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.048s user 0.005s sys 0.001s
I20260809 01:45:53.372182 10596 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_8e736088d5fe4819820a3cc50c3d1e8e" } new_table_name: "default.Table_1b84c811beb04055bfcb7e4f37bb3702" modify_external_catalogs: true
I20260809 01:45:53.416472 11951 ts_tablet_manager.cc:1929] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:53.416929 11951 log.cc:1199] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/6d35f291c92a45899eed0f2f10a3a1d8
I20260809 01:45:53.421337 12720 consensus_queue.cc:1048] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:53.422837 11554 tablet_bootstrap.cc:492] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:53.423220 11554 ts_tablet_manager.cc:1403] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.111s user 0.014s sys 0.000s
I20260809 01:45:53.417793 11951 ts_tablet_manager.cc:1950] T 6d35f291c92a45899eed0f2f10a3a1d8 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:53.431568 11553 tablet_bootstrap.cc:492] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:53.439203 11551 tablet_bootstrap.cc:492] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:53.439682 11551 ts_tablet_manager.cc:1403] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.286s user 0.013s sys 0.007s
I20260809 01:45:53.446394 11554 raft_consensus.cc:359] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.447089 11554 raft_consensus.cc:385] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:53.447348 11554 raft_consensus.cc:740] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:53.448027 11554 consensus_queue.cc:260] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.454679 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 282) took 2091 ms. Trace:
I20260809 01:45:53.455152 11951 rpcz_store.cc:276] 0809 01:45:51.362992 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.363107 (+ 115us) service_pool.cc:224] Handling call
0809 01:45:53.315969 (+1952862us) ts_tablet_manager.cc:1151] Deleting tablet 6d35f291c92a45899eed0f2f10a3a1d8
0809 01:45:53.316079 (+ 110us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.395351 (+ 79272us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.416356 (+ 21005us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.454620 (+ 38264us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1952148}
I20260809 01:45:53.457075 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 6d35f291c92a45899eed0f2f10a3a1d8 (table default.Table_14613c38aa0e4b258598254918b193bb [id=fa2e19a3042b41959165ca54e46ca690]) successfully deleted
I20260809 01:45:53.460979 10588 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_3227dde355ef4460a73ac13726bf7249" } modify_external_catalogs: true
I20260809 01:45:53.461460 10588 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_3227dde355ef4460a73ac13726bf7249" } modify_external_catalogs: true
I20260809 01:45:53.471316 11951 tablet_replica.cc:333] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:53.484570 11551 raft_consensus.cc:359] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.485221 11551 raft_consensus.cc:385] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:53.487149 11949 ts_tablet_manager.cc:1929] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:53.487563 11949 log.cc:1199] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/1bc7dc3dfc2f4c1286ca24b8e3b20719
I20260809 01:45:53.488211 11949 ts_tablet_manager.cc:1950] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:53.489141 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 258) took 4940 ms. Trace:
I20260809 01:45:53.489456 11949 rpcz_store.cc:276] 0809 01:45:48.548466 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:48.577824 (+ 29358us) service_pool.cc:224] Handling call
0809 01:45:53.182685 (+4604861us) ts_tablet_manager.cc:1151] Deleting tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719
0809 01:45:53.190454 (+ 7769us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.297399 (+106945us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.486998 (+189599us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.489084 (+ 2086us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4603814}
I20260809 01:45:53.490192 12653 raft_consensus.cc:493] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:53.490662 12653 raft_consensus.cc:515] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.492632 12653 leader_election.cc:290] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:53.494328 11554 ts_tablet_manager.cc:1434] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.071s user 0.008s sys 0.001s
I20260809 01:45:53.494529 11951 raft_consensus.cc:2243] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:53.494935 11951 raft_consensus.cc:2272] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:53.495008 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 (table default.Table_ae59ae0a1b7d4312856fd310b0c6c8b7 [id=96595f8bb6b84744bf7fc74f074a51ba]) successfully deleted
I20260809 01:45:53.495273 11554 tablet_bootstrap.cc:492] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:53.496520 11951 ts_tablet_manager.cc:1916] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:53.497550 11949 tablet_replica.cc:333] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
W20260809 01:45:53.497721 11283 leader_election.cc:343] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: dc230f25baf94cbf80aa8e2771ec23ba
W20260809 01:45:53.498378 11289 leader_election.cc:343] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: dc230f25baf94cbf80aa8e2771ec23ba
I20260809 01:45:53.498700 11289 leader_election.cc:304] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:53.499176 11289 raft_consensus.cc:2686] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 01:45:53.485474 11551 raft_consensus.cc:740] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:53.512091 11551 consensus_queue.cc:260] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.514448 11551 ts_tablet_manager.cc:1434] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.074s user 0.005s sys 0.004s
I20260809 01:45:53.524209 11551 tablet_bootstrap.cc:492] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:53.525321 11553 tablet_bootstrap.cc:654] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:53.545773 11949 raft_consensus.cc:2243] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:53.546326 11949 raft_consensus.cc:2272] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:53.548086 11949 ts_tablet_manager.cc:1916] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:53.551024 11951 ts_tablet_manager.cc:1929] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:53.551373 11951 log.cc:1199] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/dc230f25baf94cbf80aa8e2771ec23ba
I20260809 01:45:53.552022 11951 ts_tablet_manager.cc:1950] T dc230f25baf94cbf80aa8e2771ec23ba P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:53.561569 12728 raft_consensus.cc:493] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:53.562012 12728 raft_consensus.cc:515] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.563990 12728 leader_election.cc:290] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:53.564471 12728 raft_consensus.cc:493] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:53.564853 12728 raft_consensus.cc:515] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.568849 12728 leader_election.cc:290] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:53.578370 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c87361b29664464f9fd73be974e2d3a0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:53.579788 10804 leader_election.cc:343] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:53.580991 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3684fa56f9549f492a96a5c765eda78" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:53.582506 10804 leader_election.cc:343] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:39.904386 11743 server_base.cc:1061] running on GCE node
W20260809 01:45:39.916168 12151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:45:53.595381 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 283) took 2231 ms. Trace:
I20260809 01:45:53.595824 11951 rpcz_store.cc:276] 0809 01:45:51.364239 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.364369 (+ 130us) service_pool.cc:224] Handling call
0809 01:45:53.457873 (+2093504us) ts_tablet_manager.cc:1151] Deleting tablet dc230f25baf94cbf80aa8e2771ec23ba
0809 01:45:53.471204 (+ 13331us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.507447 (+ 36243us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.550905 (+ 43458us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.595314 (+ 44409us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22352,"tablet-delete.queue_time_us":2092824}
I20260809 01:45:53.597250 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet dc230f25baf94cbf80aa8e2771ec23ba (table default.Table_1998ec81597a4c4584dc4e4f90caa274 [id=cb347b72c1cf436095c6043c5b169641]) successfully deleted
I20260809 01:45:53.597596 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3684fa56f9549f492a96a5c765eda78" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:53.598167 11951 tablet_replica.cc:333] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:53.598876 11951 raft_consensus.cc:2243] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:53.599941 11951 raft_consensus.cc:2272] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:53.601065 10805 leader_election.cc:343] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:53.601432 11951 ts_tablet_manager.cc:1916] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:53.601408 10805 leader_election.cc:304] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:53.602669 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c87361b29664464f9fd73be974e2d3a0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:53.605650 11554 tablet_bootstrap.cc:654] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:53.608230 10805 leader_election.cc:343] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:53.608569 10805 leader_election.cc:304] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:53.609491 12728 raft_consensus.cc:2749] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:53.610150 12728 raft_consensus.cc:2749] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:53.610481 10595 master_service.cc:960] ReplaceTablet: received request to replace tablet a85e82d85424445d94b30402d41c53dc from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:53.617404 11551 tablet_bootstrap.cc:654] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:53.620146 12826 raft_consensus.cc:493] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:53.629827 10596 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a9c3b06f67954ba68edcf505118eaaec has no consensus state
I20260809 01:45:53.629505 12826 raft_consensus.cc:515] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:39.683362 12143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:45:53.635833 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7a8b6d06064addb02979a7feedf756" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:53.636193 11554 tablet_bootstrap.cc:492] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:53.636209 11743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:45:53.636562 11554 ts_tablet_manager.cc:1403] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.141s user 0.010s sys 0.001s
W20260809 01:45:53.637241 10804 leader_election.cc:343] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:53.638101 12826 leader_election.cc:290] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:53.639137 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7a8b6d06064addb02979a7feedf756" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:53.644081 11553 tablet_bootstrap.cc:492] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:53.644608 11553 ts_tablet_manager.cc:1403] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.213s user 0.008s sys 0.003s
W20260809 01:45:53.646323 10805 leader_election.cc:343] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:53.646708 10805 leader_election.cc:304] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:53.647557 11553 raft_consensus.cc:359] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.648170 11553 raft_consensus.cc:385] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:53.648422 11553 raft_consensus.cc:740] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:53.652261 11554 raft_consensus.cc:359] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.652978 11554 raft_consensus.cc:385] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:53.653292 11554 raft_consensus.cc:740] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:53.654063 11554 consensus_queue.cc:260] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.656335 10596 master_service.cc:960] ReplaceTablet: received request to replace tablet 4aa1429f21024739b06bec88c3cb24d9 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:53.657001 12826 raft_consensus.cc:2749] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:53.659852 12653 raft_consensus.cc:493] T 83ac8498dff04b4e93303d7a35e62134 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:53.659349 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_447d9518a2a7484ba63a2950dea459df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:53.660249 12653 raft_consensus.cc:515] T 83ac8498dff04b4e93303d7a35e62134 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:53.661326 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_447d9518a2a7484ba63a2950dea459df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:53.662189 11551 tablet_bootstrap.cc:492] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:53.662151 12653 leader_election.cc:290] T 83ac8498dff04b4e93303d7a35e62134 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:53.662658 11551 ts_tablet_manager.cc:1403] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.139s user 0.008s sys 0.001s
I20260809 01:45:53.663949 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac8498dff04b4e93303d7a35e62134" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:53.666453 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac8498dff04b4e93303d7a35e62134" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:53.666385 11551 raft_consensus.cc:359] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.666989 11551 raft_consensus.cc:385] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:53.667249 11551 raft_consensus.cc:740] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:53.667956 11289 leader_election.cc:304] T 83ac8498dff04b4e93303d7a35e62134 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:53.667866 11551 consensus_queue.cc:260] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.668917 12653 raft_consensus.cc:2749] T 83ac8498dff04b4e93303d7a35e62134 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:53.669826 11551 ts_tablet_manager.cc:1434] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 01:45:53.670750 11551 tablet_bootstrap.cc:492] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:53.678673 11743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:45:53.682729 12826 raft_consensus.cc:493] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:53.683204 12826 raft_consensus.cc:515] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.684054 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_ba11ce5b2eda44f897a3689070a56b40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:53.685025 12826 leader_election.cc:290] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:53.685806 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba11ce5b2eda44f897a3689070a56b40 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:53.686174 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6c244108a4ff3a2df0585e5280294" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:53.686592 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6c244108a4ff3a2df0585e5280294" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:53.687567 10805 leader_election.cc:343] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:53.687949 10804 leader_election.cc:343] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:53.688308 10804 leader_election.cc:304] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:53.689173 12826 raft_consensus.cc:2749] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:53.693691 10579 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet a9c3b06f67954ba68edcf505118eaaec (table default.Table_1b84c811beb04055bfcb7e4f37bb3702 [id=b328ad295fa1411b872de319d21eeb1c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a9c3b06f67954ba68edcf505118eaaec has no consensus state
I20260809 01:45:53.699592 11743 hybrid_clock.cc:648] HybridClock initialized: now 1786239953699444 us; error 128 us; skew 500 ppm
I20260809 01:45:53.699909 11949 ts_tablet_manager.cc:1929] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:53.700258 11949 log.cc:1199] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/9b51f999fab24b1ca7451ff543a124a4
I20260809 01:45:53.700985 11949 ts_tablet_manager.cc:1950] T 9b51f999fab24b1ca7451ff543a124a4 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:53.701467 11743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:45:53.704734 11551 tablet_bootstrap.cc:654] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:53.710729 11554 ts_tablet_manager.cc:1434] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.074s user 0.003s sys 0.003s
I20260809 01:45:53.710634 11553 consensus_queue.cc:260] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.719030 11554 tablet_bootstrap.cc:492] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:53.721067 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_01a686662b59496fa2cbde4a0304f124"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:53.724824 11554 tablet_bootstrap.cc:654] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:53.726649 11551 tablet_bootstrap.cc:492] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:53.727027 11551 ts_tablet_manager.cc:1403] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.056s user 0.005s sys 0.005s
I20260809 01:45:53.730316 11551 raft_consensus.cc:359] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.730945 11551 raft_consensus.cc:385] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:53.731189 11551 raft_consensus.cc:740] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:53.731748 11551 consensus_queue.cc:260] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.736826 12653 raft_consensus.cc:493] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:53.737254 12653 raft_consensus.cc:515] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.739197 12653 leader_election.cc:290] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:53.739380 11551 ts_tablet_manager.cc:1434] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260809 01:45:53.740177 12826 raft_consensus.cc:493] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:53.740221 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01a686662b59496fa2cbde4a0304f124 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:53.740598 12826 raft_consensus.cc:515] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:53.741950 11289 leader_election.cc:343] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: b84f1943c1a34e999fc7c12a5847cdb6
I20260809 01:45:53.742488 12826 leader_election.cc:290] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:53.744266 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81da8d08131d4a3ba636ecfc7a7d2951" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:53.745656 11551 tablet_bootstrap.cc:492] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
W20260809 01:45:53.757889 10805 leader_election.cc:343] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:53.767045 11951 ts_tablet_manager.cc:1929] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:53.767383 11951 log.cc:1199] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/c2f86bba4bd94fbcacdc9c139e9de2d7
I20260809 01:45:53.767952 11951 ts_tablet_manager.cc:1950] T c2f86bba4bd94fbcacdc9c139e9de2d7 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:53.769671 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 284) took 2403 ms. Trace:
I20260809 01:45:53.769984 11951 rpcz_store.cc:276] 0809 01:45:51.366555 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.366674 (+ 119us) service_pool.cc:224] Handling call
0809 01:45:53.598026 (+2231352us) ts_tablet_manager.cc:1151] Deleting tablet c2f86bba4bd94fbcacdc9c139e9de2d7
0809 01:45:53.598066 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.652467 (+ 54401us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.766932 (+114465us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.769613 (+ 2681us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2230621}
I20260809 01:45:53.771149 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet c2f86bba4bd94fbcacdc9c139e9de2d7 (table default.Table_5f4d617a86c2487fa9f64cb58f14de3c [id=f64daa5a2706436081e9166a5f3a0cd7]) successfully deleted
I20260809 01:45:53.782825 11743 webserver.cc:533] Webserver started at http://127.10.8.125:46803/ using document root <none> and password file <none>
I20260809 01:45:53.784327 11743 fs_manager.cc:362] Metadata directory not provided
I20260809 01:45:53.784650 11743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:45:53.786756 11553 ts_tablet_manager.cc:1434] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.142s user 0.003s sys 0.003s
I20260809 01:45:53.789112 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_f4a23e78e1d54205b4e1d03c5dd16fbe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:53.791051 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4a23e78e1d54205b4e1d03c5dd16fbe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:53.796007 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 288) took 2290 ms. Trace:
I20260809 01:45:53.796298 11384 raft_consensus.cc:3060] T 83ac8498dff04b4e93303d7a35e62134 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:53.796320 11951 rpcz_store.cc:276] 0809 01:45:51.505594 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.514651 (+ 9057us) service_pool.cc:224] Handling call
0809 01:45:53.795642 (+2280991us) ts_tablet_manager.cc:1151] Deleting tablet 991e8580b8624f34881d56c6cc60600f
0809 01:45:53.795691 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.795950 (+ 259us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2280229}
I20260809 01:45:53.798184 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 290) took 2031 ms. Trace:
W20260809 01:45:53.798286 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 991e8580b8624f34881d56c6cc60600f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 991e8580b8624f34881d56c6cc60600f already in progress: creating tablet
I20260809 01:45:53.798501 11951 rpcz_store.cc:276] 0809 01:45:51.766449 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.766559 (+ 110us) service_pool.cc:224] Handling call
0809 01:45:53.797763 (+2031204us) ts_tablet_manager.cc:1151] Deleting tablet 7634fe763a4e4fec91c5938e45a5d1fc
0809 01:45:53.797915 (+ 152us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.798136 (+ 221us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1957847}
I20260809 01:45:53.798705 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 259) took 5249 ms. Trace:
I20260809 01:45:53.799556 11951 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 301) took 1313 ms. Trace:
I20260809 01:45:53.799168 11949 rpcz_store.cc:276] 0809 01:45:48.548786 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:48.584294 (+ 35508us) service_pool.cc:224] Handling call
0809 01:45:53.490515 (+4906221us) ts_tablet_manager.cc:1151] Deleting tablet 9b51f999fab24b1ca7451ff543a124a4
0809 01:45:53.490553 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.579945 (+ 89392us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.699782 (+119837us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:53.798640 (+ 98858us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3089,"tablet-delete.queue_time_us":4905320}
W20260809 01:45:53.799711 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 7634fe763a4e4fec91c5938e45a5d1fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7634fe763a4e4fec91c5938e45a5d1fc already in progress: creating tablet
I20260809 01:45:53.800333 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 9b51f999fab24b1ca7451ff543a124a4 (table default.Table_5264c640dfc04d9bae010805cdaf4ea6 [id=c7b50f12c7934944aceb691a5d7b0609]) successfully deleted
I20260809 01:45:53.800787 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 260) took 4956 ms. Trace:
I20260809 01:45:53.801087 11949 rpcz_store.cc:276] 0809 01:45:48.844724 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:48.844886 (+ 162us) service_pool.cc:224] Handling call
0809 01:45:53.800465 (+4955579us) ts_tablet_manager.cc:1151] Deleting tablet d146bdaf4dd2418ca870c4afeb6ccb6c
0809 01:45:53.800510 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.800734 (+ 224us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4954917}
W20260809 01:45:53.802356 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet d146bdaf4dd2418ca870c4afeb6ccb6c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d146bdaf4dd2418ca870c4afeb6ccb6c already in progress: creating tablet
I20260809 01:45:53.802625 11949 tablet_replica.cc:333] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:53.799831 11951 rpcz_store.cc:276] 0809 01:45:52.485879 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:52.485982 (+ 103us) service_pool.cc:224] Handling call
0809 01:45:53.799342 (+1313360us) ts_tablet_manager.cc:1151] Deleting tablet 4754f2e2a35f4a6a91fa333e2cf564e4
0809 01:45:53.799378 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:53.799509 (+ 131us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1303470}
I20260809 01:45:53.803488 11949 raft_consensus.cc:2243] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:53.804157 11949 raft_consensus.cc:2272] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:53.804733 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 4754f2e2a35f4a6a91fa333e2cf564e4 because the tablet was not found. No further retry: Not found: Tablet not found: 4754f2e2a35f4a6a91fa333e2cf564e4
W20260809 01:45:53.805310 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet b9493892c1ff48e7a0558ff0fcb7a0f3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9493892c1ff48e7a0558ff0fcb7a0f3 already in progress: creating tablet
W20260809 01:45:53.806515 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet b095338eb59e4161aaaf3afb762cd296 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b095338eb59e4161aaaf3afb762cd296 already in progress: creating tablet
I20260809 01:45:53.833950 10591 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_85b1c71bf0984d6bb91bbc04608af545" } new_table_name: "default.Table_3dd6aaf3d0d545529f081db48de38cc2" modify_external_catalogs: true
I20260809 01:45:53.844460 11553 tablet_bootstrap.cc:492] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:53.839354 12826 raft_consensus.cc:493] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:53.848744 12826 raft_consensus.cc:515] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.850684 12826 leader_election.cc:290] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:53.852478 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd3662c307ba4ff78b8f3165b23acb1a" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:53.853853 10805 leader_election.cc:343] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:53.859167 11743 fs_manager.cc:714] Time spent opening directory manager: real 0.044s user 0.003s sys 0.005s
W20260809 01:45:53.866894 10591 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 60176118d61548ff96b001ab0dae7d21 consensus state has no leader
I20260809 01:45:53.879295 12847 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:45:53.886788 11743 fs_manager.cc:730] Time spent opening block manager: real 0.024s user 0.001s sys 0.005s
I20260809 01:45:53.887311 11743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/data,/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/wal
uuid: "71bc72deb1d2402cabd817dfe3b27e88"
format_stamp: "Formatted at 2026-08-09 01:45:13 on dist-test-slave-zr1t"
I20260809 01:45:53.891952 11743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:45:53.893260 11553 tablet_bootstrap.cc:654] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:53.894009 11949 ts_tablet_manager.cc:1916] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:53.901077 11554 tablet_bootstrap.cc:492] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:53.901551 11554 ts_tablet_manager.cc:1403] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.183s user 0.006s sys 0.043s
I20260809 01:45:53.901400 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_882931db60674255a2de681747e0a21e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:53.903177 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_882931db60674255a2de681747e0a21e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:53.903985 11554 raft_consensus.cc:359] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.904560 11554 raft_consensus.cc:385] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:53.904809 11554 raft_consensus.cc:740] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:53.917161 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81da8d08131d4a3ba636ecfc7a7d2951" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:53.919858 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd3662c307ba4ff78b8f3165b23acb1a" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:53.927300 10804 leader_election.cc:343] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:53.927778 11283 leader_election.cc:343] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: b84f1943c1a34e999fc7c12a5847cdb6
I20260809 01:45:53.927769 10804 leader_election.cc:304] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:53.928380 10804 raft_consensus.cc:2686] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 01:45:53.928195 11283 leader_election.cc:304] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:53.929111 10804 leader_election.cc:343] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:53.929216 12653 raft_consensus.cc:2749] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:53.929476 10804 leader_election.cc:304] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:53.930449 11554 consensus_queue.cc:260] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.931135 12826 raft_consensus.cc:2749] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:53.942890 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_f79203f002e14b909a945be94e3e4782"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:53.944690 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f79203f002e14b909a945be94e3e4782 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:53.950024 12653 raft_consensus.cc:493] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:53.950505 12653 raft_consensus.cc:515] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:53.952515 12653 leader_election.cc:290] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:53.954909 11289 leader_election.cc:343] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 36a542c773b7438ead16c5b3af224cf9
W20260809 01:45:53.957561 11283 leader_election.cc:343] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: 36a542c773b7438ead16c5b3af224cf9
I20260809 01:45:53.957903 11283 leader_election.cc:304] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:53.958817 12653 raft_consensus.cc:2749] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:53.971274 10597 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 75e4315b3c644846ac572883c51f23de (table default.Table_ee7c1665ed2344278d1ee031c67438a4 [id=850f362a7f9a4531a3d7d6de29b695ba]). Expected version 4 got 0
I20260809 01:45:53.976163 10589 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b (table default.Table_0bcdcc37d5f04f40ab4dd02b66baf4a6 [id=97ccdb35be6a437288222feb89f9e751]). Expected version 1 got 0
I20260809 01:45:53.986292 11554 ts_tablet_manager.cc:1434] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.084s user 0.004s sys 0.001s
I20260809 01:45:53.998605 11554 tablet_bootstrap.cc:492] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:54.000087 12826 raft_consensus.cc:493] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:54.000579 12826 raft_consensus.cc:515] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.002673 12826 leader_election.cc:290] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:54.004459 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938bfd21b294fa6a25f0d8efc1ca29a" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:54.006064 10805 leader_election.cc:343] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:54.007196 11551 tablet_bootstrap.cc:654] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:54.007638 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938bfd21b294fa6a25f0d8efc1ca29a" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:54.012465 11334 tablet_service.cc:1511] Processing CreateTablet for tablet 4b976164fd514af28ac8f97d26d83651 (DEFAULT_TABLE table=default.Table_e755d806f33849768faa61b4cc03b181 [id=930cfb0a66d5493ca68b5cc842bf2b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.013795 11334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b976164fd514af28ac8f97d26d83651. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:54.018885 10804 leader_election.cc:343] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:54.019268 10804 leader_election.cc:304] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:54.021720 12826 raft_consensus.cc:2749] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:54.022692 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 4b976164fd514af28ac8f97d26d83651 (DEFAULT_TABLE table=default.Table_e755d806f33849768faa61b4cc03b181 [id=930cfb0a66d5493ca68b5cc842bf2b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.023147 11554 tablet_bootstrap.cc:654] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:54.023730 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b976164fd514af28ac8f97d26d83651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.032984 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 83e67f30c6df4d4ab05c58a138a70c9a (DEFAULT_TABLE table=default.Table_65be7c5fe96148898b9fdb468cee042d [id=26965350b92d4238ac18d1405c31b436]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.034163 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83e67f30c6df4d4ab05c58a138a70c9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.037140 10877 tablet_service.cc:1511] Processing CreateTablet for tablet 83e67f30c6df4d4ab05c58a138a70c9a (DEFAULT_TABLE table=default.Table_65be7c5fe96148898b9fdb468cee042d [id=26965350b92d4238ac18d1405c31b436]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.038223 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83e67f30c6df4d4ab05c58a138a70c9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.047925 11337 tablet_service.cc:1511] Processing CreateTablet for tablet e0fdf4f4b1d44979936b37886bc60836 (DEFAULT_TABLE table=default.Table_8dde73a0e7f44af28232bbb7a0d57897 [id=cf06c89e274a4b81b578c53aaf7cd80a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.048884 11337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0fdf4f4b1d44979936b37886bc60836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.050982 10852 tablet_service.cc:1511] Processing CreateTablet for tablet e0fdf4f4b1d44979936b37886bc60836 (DEFAULT_TABLE table=default.Table_8dde73a0e7f44af28232bbb7a0d57897 [id=cf06c89e274a4b81b578c53aaf7cd80a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.052033 10852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0fdf4f4b1d44979936b37886bc60836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.058189 11334 tablet_service.cc:1511] Processing CreateTablet for tablet d8e45cd77b334bdf9948c20ff3ff764c (DEFAULT_TABLE table=default.Table_a5ce279641de467da9670256492f78e2 [id=13eb3e4fa11e4dd493b69b01f07deafc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.062656 11334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e45cd77b334bdf9948c20ff3ff764c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.066104 12599 raft_consensus.cc:493] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:54.066777 12599 raft_consensus.cc:515] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.069088 12599 leader_election.cc:290] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:54.069576 10873 tablet_service.cc:1511] Processing CreateTablet for tablet d8e45cd77b334bdf9948c20ff3ff764c (DEFAULT_TABLE table=default.Table_a5ce279641de467da9670256492f78e2 [id=13eb3e4fa11e4dd493b69b01f07deafc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.070571 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e45cd77b334bdf9948c20ff3ff764c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:54.072467 11063 leader_election.cc:343] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 197b5043f6d348f89853c073b256daa4
I20260809 01:45:54.073412 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197b5043f6d348f89853c073b256daa4" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:54.074602 12599 raft_consensus.cc:493] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:54.075062 12599 raft_consensus.cc:515] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:54.075785 11062 leader_election.cc:343] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:54.076160 11062 leader_election.cc:304] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:54.077222 11351 tablet_service.cc:1511] Processing CreateTablet for tablet 7c3f7eefb1f649d8a34f2c5de71d361e (DEFAULT_TABLE table=default.Table_65902a8894274e2ca34e86aa908b12f4 [id=b4d4214e9b9942ec87c7f2aaea9991ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.078285 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c3f7eefb1f649d8a34f2c5de71d361e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:54.078727 11063 leader_election.cc:343] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: b2726cb09f734ed18bb9379f94f3f22a
I20260809 01:45:54.078825 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2726cb09f734ed18bb9379f94f3f22a" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:54.079167 12599 leader_election.cc:290] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:54.081271 12599 raft_consensus.cc:2749] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:54.083025 11062 leader_election.cc:343] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:54.083436 11062 leader_election.cc:304] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:54.085487 11383 raft_consensus.cc:3060] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:54.085434 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_88a5a7cfe7174a30bc65b1943fc5e8b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:54.086992 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_88a5a7cfe7174a30bc65b1943fc5e8b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:54.094130 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_1417e8aa601a422ea188f62cd2c3642f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:54.095664 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1417e8aa601a422ea188f62cd2c3642f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:54.099635 10855 tablet_service.cc:1511] Processing CreateTablet for tablet 7c3f7eefb1f649d8a34f2c5de71d361e (DEFAULT_TABLE table=default.Table_65902a8894274e2ca34e86aa908b12f4 [id=b4d4214e9b9942ec87c7f2aaea9991ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.105181 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c3f7eefb1f649d8a34f2c5de71d361e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.105661 12653 raft_consensus.cc:493] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:54.106052 12653 raft_consensus.cc:515] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.109170 12826 raft_consensus.cc:493] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:54.109637 12826 raft_consensus.cc:515] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.109863 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb9afd3be194830ba99f5e20fdaca0a" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:54.110441 10910 raft_consensus.cc:2468] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:54.111601 12826 leader_election.cc:290] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:54.114174 11289 leader_election.cc:304] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:54.116420 12864 raft_consensus.cc:2804] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 01:45:54.116828 12864 raft_consensus.cc:493] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:54.117213 12864 raft_consensus.cc:3060] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Advancing to term 2
I20260809 01:45:54.120529 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b297f9083744e2953d41cd7b1055df" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:54.122473 10805 leader_election.cc:343] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:54.122704 11351 tablet_service.cc:1511] Processing CreateTablet for tablet 62405c58875b433da4fb2d1cbcae1ab4 (DEFAULT_TABLE table=default.Table_55937d2e689c4822b6d1c366784f19cf [id=7408c688fcd5447d802070b8412e7837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.128329 10858 tablet_service.cc:1511] Processing CreateTablet for tablet 62405c58875b433da4fb2d1cbcae1ab4 (DEFAULT_TABLE table=default.Table_55937d2e689c4822b6d1c366784f19cf [id=7408c688fcd5447d802070b8412e7837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.125073 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62405c58875b433da4fb2d1cbcae1ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.129331 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62405c58875b433da4fb2d1cbcae1ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.129938 12826 raft_consensus.cc:493] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:54.130511 12826 raft_consensus.cc:515] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.132793 12826 leader_election.cc:290] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:54.138154 11334 tablet_service.cc:1511] Processing CreateTablet for tablet bc3e7e481fcb484bbeedf3e48d6b02c1 (DEFAULT_TABLE table=default.Table_197551df65a34e48be825222e02fc1cc [id=ec37d1dc50894321b6efecde89663c5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.139060 11334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc3e7e481fcb484bbeedf3e48d6b02c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.140703 10856 tablet_service.cc:1511] Processing CreateTablet for tablet bc3e7e481fcb484bbeedf3e48d6b02c1 (DEFAULT_TABLE table=default.Table_197551df65a34e48be825222e02fc1cc [id=ec37d1dc50894321b6efecde89663c5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.140892 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a5ef5a3ff44f74ab19998ba9f19310" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:54.141989 10856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc3e7e481fcb484bbeedf3e48d6b02c1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:54.142076 10805 leader_election.cc:343] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:54.145565 12858 tablet.cc:1724] T cef88ffe9e95437da81793795742def3 P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:54.150249 11341 tablet_service.cc:1511] Processing CreateTablet for tablet e943a4fa04e542e5bcd82ee2ce022642 (DEFAULT_TABLE table=default.Table_c770b2448fca4145918eed5aa2fbc2f6 [id=24334f346e0a4bb89fd00d9758f71045]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.151386 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e943a4fa04e542e5bcd82ee2ce022642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.154654 12864 raft_consensus.cc:515] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.156524 10864 tablet_service.cc:1511] Processing CreateTablet for tablet e943a4fa04e542e5bcd82ee2ce022642 (DEFAULT_TABLE table=default.Table_c770b2448fca4145918eed5aa2fbc2f6 [id=24334f346e0a4bb89fd00d9758f71045]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.157881 10864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e943a4fa04e542e5bcd82ee2ce022642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.159775 11366 tablet_service.cc:1511] Processing CreateTablet for tablet f44b03d0f5b4480ebf4a7f1d874820c7 (DEFAULT_TABLE table=default.Table_9f368169db0b40ee91c03bdc0b8d4785 [id=68875f8f88624a999bd31162c3300b0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.160663 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f44b03d0f5b4480ebf4a7f1d874820c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.175179 10870 tablet_service.cc:1511] Processing CreateTablet for tablet f44b03d0f5b4480ebf4a7f1d874820c7 (DEFAULT_TABLE table=default.Table_9f368169db0b40ee91c03bdc0b8d4785 [id=68875f8f88624a999bd31162c3300b0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.176285 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f44b03d0f5b4480ebf4a7f1d874820c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.184975 10865 tablet_service.cc:1511] Processing CreateTablet for tablet 46bbf262a261468da91230a3ca68cbea (DEFAULT_TABLE table=default.Table_cc3003c3373b4f6cbcae8eb85030466f [id=b33ab68357c0431ca78bdc90bda6a7d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.186347 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46bbf262a261468da91230a3ca68cbea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.203027 12864 leader_election.cc:290] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:54.204444 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 46bbf262a261468da91230a3ca68cbea (DEFAULT_TABLE table=default.Table_cc3003c3373b4f6cbcae8eb85030466f [id=b33ab68357c0431ca78bdc90bda6a7d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.205503 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46bbf262a261468da91230a3ca68cbea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.209019 10873 tablet_service.cc:1511] Processing CreateTablet for tablet acc9084f4ceb479d85dbc9146bb43c8f (DEFAULT_TABLE table=default.Table_b2acb056eb544b64b517a5b0fdd3f131 [id=83da413be8d14f2e819cd6f2ca0befe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.210861 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb9afd3be194830ba99f5e20fdaca0a" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:54.212411 10908 raft_consensus.cc:3060] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Advancing to term 2
I20260809 01:45:54.218400 10858 tablet_service.cc:1511] Processing CreateTablet for tablet 4cc185f76b12407fa0da8d6a96e7a06e (DEFAULT_TABLE table=default.Table_5f7c40d82c854aaa87deac85836396ca [id=e46576dcc09c48c08c346055f67cc701]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.219416 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cc185f76b12407fa0da8d6a96e7a06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.220669 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b297f9083744e2953d41cd7b1055df" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:54.223663 11154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a5ef5a3ff44f74ab19998ba9f19310" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:54.223978 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb9afd3be194830ba99f5e20fdaca0a" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:54.225080 12778 raft_consensus.cc:2749] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:54.225458 12720 consensus_queue.cc:1048] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:45:54.212481 11334 tablet_service.cc:1511] Processing CreateTablet for tablet 4cc185f76b12407fa0da8d6a96e7a06e (DEFAULT_TABLE table=default.Table_5f7c40d82c854aaa87deac85836396ca [id=e46576dcc09c48c08c346055f67cc701]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.231555 11334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cc185f76b12407fa0da8d6a96e7a06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.227178 11346 tablet_service.cc:1511] Processing CreateTablet for tablet acc9084f4ceb479d85dbc9146bb43c8f (DEFAULT_TABLE table=default.Table_b2acb056eb544b64b517a5b0fdd3f131 [id=83da413be8d14f2e819cd6f2ca0befe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.235657 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc9084f4ceb479d85dbc9146bb43c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.236581 11351 tablet_service.cc:1511] Processing CreateTablet for tablet 6134c0e8fa79403b8785dda379c01a0f (DEFAULT_TABLE table=default.Table_0ccb4fdb37aa49a2bf24e84e07a54081 [id=7d3758d6271943b3b7b78eed034ec422]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.237926 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6134c0e8fa79403b8785dda379c01a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.230612 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb9afd3be194830ba99f5e20fdaca0a" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:54.245414 11383 raft_consensus.cc:1240] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [term 2 FOLLOWER]: Rejecting Update request from peer 33b5652538e64ed58a09993d3188f8d7 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260809 01:45:54.247063 12720 consensus_queue.cc:1059] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:54.211800 12860 tablet.cc:1724] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:54.252081 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 74d6b03590c8472db8977bee5962aa6c (DEFAULT_TABLE table=default.Table_eb6df7e248d44109898ab900441bda8b [id=7701d6a8eff047ddac4f8775385c57ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.252950 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74d6b03590c8472db8977bee5962aa6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.253007 11337 tablet_service.cc:1511] Processing CreateTablet for tablet d2baec6d21e24c458bfbc2824408b5c4 (DEFAULT_TABLE table=default.Table_cfd1f608be9041d185710895f202f65e [id=d41a174cc20a4a6db3f8ae3b8622f48e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.253877 11337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2baec6d21e24c458bfbc2824408b5c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.254096 11347 tablet_service.cc:1511] Processing CreateTablet for tablet fd8c5a45dec34006bd9c9894d336815c (DEFAULT_TABLE table=default.Table_99228426aa9e4d1b84a5f23af6de8669 [id=9b698cf563244d5c87b04054906bde65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.254269 11369 tablet_service.cc:1511] Processing CreateTablet for tablet cb13f4c7b07348cba67b6b43a9b154bd (DEFAULT_TABLE table=default.Table_ce7493c5349e41d890087f1336689d4a [id=393bf4cab7eb48859853968aeae87b42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.254956 11347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd8c5a45dec34006bd9c9894d336815c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.255510 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb13f4c7b07348cba67b6b43a9b154bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.268714 11551 tablet_bootstrap.cc:492] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:54.269158 11551 ts_tablet_manager.cc:1403] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.529s user 0.011s sys 0.003s
I20260809 01:45:54.270005 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 7eb2acab69734479aa729c65666b9081 (DEFAULT_TABLE table=default.Table_9e3630f9d80845d5bc57f0dde0550a41 [id=c250aefe7f6c49888699e646b31eae39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.270879 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eb2acab69734479aa729c65666b9081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.280159 12720 raft_consensus.cc:3055] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 01:45:54.213847 12653 leader_election.cc:290] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:54.280525 12720 raft_consensus.cc:740] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:54.281687 12720 consensus_queue.cc:260] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.282713 12720 raft_consensus.cc:3060] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Advancing to term 2
I20260809 01:45:54.284456 11345 tablet_service.cc:1511] Processing CreateTablet for tablet f7d7a8f179244b3b9c9ca042af67e4c0 (DEFAULT_TABLE table=default.Table_836b8d3fc2e34cb786cadfc1794cb1ad [id=7b4191ad61c143eb8badf6c4f983ecee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.285578 11345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7d7a8f179244b3b9c9ca042af67e4c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.290436 12599 raft_consensus.cc:493] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:54.290902 11369 tablet_service.cc:1511] Processing CreateTablet for tablet f4985b1715f64adaae432d46dee4ea38 (DEFAULT_TABLE table=default.Table_8060ad8eb1be43f4a3c2e9dac705c0e3 [id=d3b5229e374c4e6bbce3104b99c494d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.290916 12599 raft_consensus.cc:515] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.291872 11369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4985b1715f64adaae432d46dee4ea38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.294867 12599 leader_election.cc:290] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:54.307590 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f022e6a57a4ec0a5a8c33159e97a8e" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:54.309333 11062 leader_election.cc:343] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:54.315701 10592 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 2790e2266ade4c698741a940ed49757e
I20260809 01:45:54.317293 10592 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 392) took 1012 ms. Trace:
I20260809 01:45:54.317670 11337 tablet_service.cc:1558] Processing DeleteTablet for tablet 2790e2266ade4c698741a940ed49757e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:53 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:54.318025 10592 rpcz_store.cc:276] 0809 01:45:53.305069 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:53.305187 (+ 118us) service_pool.cc:224] Handling call
0809 01:45:53.306446 (+ 1259us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:53.307050 (+ 604us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:53.307168 (+ 118us) catalog_manager.cc:2850] Locking tablets
0809 01:45:53.963369 (+656201us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:54.315347 (+351978us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:54.315539 (+ 192us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:54.315620 (+ 81us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:54.317233 (+ 1613us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:53.965098 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:53.965583 (+ 485us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:53.965610 (+ 27us) write_op.cc:435] Acquired schema lock
0809 01:45:53.965626 (+ 16us) tablet.cc:662] Decoding operations
0809 01:45:53.965982 (+ 356us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:53.966025 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:53.966049 (+ 24us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:53.966369 (+ 320us) tablet.cc:701] Row locks acquired
0809 01:45:53.966383 (+ 14us) write_op.cc:260] PREPARE: finished
0809 01:45:53.966515 (+ 132us) write_op.cc:270] Start()
0809 01:45:53.966606 (+ 91us) write_op.cc:276] Timestamp: P: 1786239953966430 usec, L: 0
0809 01:45:53.966620 (+ 14us) op_driver.cc:348] REPLICATION: starting
0809 01:45:54.098175 (+131555us) log.cc:844] Serialized 725 byte log entry
0809 01:45:54.099043 (+ 868us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678d951 00007fa2c6820208 00007fa2c6caff61 00007fa2c6caf738 00007fa2c6cb43d1 00007fa2c6cb4389 00007fa2c6cb4319 00007fa2c6cb42e1 00007fa2c6cb350d 00000000005bd8c4 00000000005bd6f9
0809 01:45:54.313691 (+214648us) op_driver.cc:464] REPLICATION: finished
0809 01:45:54.314150 (+ 459us) write_op.cc:301] APPLY: starting
0809 01:45:54.314203 (+ 53us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:54.314293 (+ 90us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:54.314313 (+ 20us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:54.314546 (+ 233us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:54.314675 (+ 129us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:54.314715 (+ 40us) write_op.cc:312] APPLY: finished
0809 01:45:54.314911 (+ 196us) log.cc:844] Serialized 29 byte log entry
0809 01:45:54.315066 (+ 155us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:54.315127 (+ 61us) write_op.cc:454] Released schema lock
0809 01:45:54.315160 (+ 33us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":326,"num_ops":2,"prepare.queue_time_us":113,"prepare.run_cpu_time_us":4635,"prepare.run_wall_time_us":134129,"raft.queue_time_us":2483,"raft.run_cpu_time_us":995,"raft.run_wall_time_us":996,"replication_time_us":346981,"spinlock_wait_cycles":301287168}]]}
I20260809 01:45:54.324501 11336 tablet_service.cc:1511] Processing CreateTablet for tablet 5e9afec0618e4fa293f6dad6769ae49c (DEFAULT_TABLE table=default.Table_1391d3ee4c884a43b3fa55ce4cf9d7d4 [id=0b559c6e6c364d2ab92c981378dc2719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.325644 11336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e9afec0618e4fa293f6dad6769ae49c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.326002 11341 tablet_service.cc:1511] Processing CreateTablet for tablet 878b7a9d92df4824902b2b15295f65df (DEFAULT_TABLE table=default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.327045 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 878b7a9d92df4824902b2b15295f65df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.336933 10596 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 4aa1429f21024739b06bec88c3cb24d9
I20260809 01:45:54.338505 10596 catalog_manager.cc:6807] ReplaceTablet: tablet 4aa1429f21024739b06bec88c3cb24d9 deleted and replaced by tablet bdde640ea8c04e8381c71acae7ac809f
I20260809 01:45:54.356316 11360 tablet_service.cc:1558] Processing DeleteTablet for tablet 4aa1429f21024739b06bec88c3cb24d9 with delete_type TABLET_DATA_DELETED (replaced by tablet bdde640ea8c04e8381c71acae7ac809f) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:54.356567 11349 tablet_service.cc:1511] Processing CreateTablet for tablet 1b25a1da87fb445e8bcf9468862ae976 (DEFAULT_TABLE table=default.Table_779631fdff574e96846c0dd384b81c9f [id=b289c25942334fd8946e86111451d9d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.358059 11349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b25a1da87fb445e8bcf9468862ae976. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:54.359301 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 2790e2266ade4c698741a940ed49757e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2790e2266ade4c698741a940ed49757e already in progress: creating tablet
I20260809 01:45:54.362156 11360 tablet_service.cc:1511] Processing CreateTablet for tablet c7f9dd670ee146bea3cedffea82a91cc (DEFAULT_TABLE table=default.Table_51519b5a61474dba91f80b33789a2849 [id=aa77285868874c4db4fea36ac70035b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.363142 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f9dd670ee146bea3cedffea82a91cc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:54.371950 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 4aa1429f21024739b06bec88c3cb24d9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4aa1429f21024739b06bec88c3cb24d9 already in progress: creating tablet
I20260809 01:45:54.391865 12778 raft_consensus.cc:493] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:54.392405 12778 raft_consensus.cc:515] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.393227 11125 tablet_service.cc:1511] Processing CreateTablet for tablet 83e67f30c6df4d4ab05c58a138a70c9a (DEFAULT_TABLE table=default.Table_65be7c5fe96148898b9fdb468cee042d [id=26965350b92d4238ac18d1405c31b436]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.394682 11125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83e67f30c6df4d4ab05c58a138a70c9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.394532 11113 tablet_service.cc:1511] Processing CreateTablet for tablet 4b976164fd514af28ac8f97d26d83651 (DEFAULT_TABLE table=default.Table_e755d806f33849768faa61b4cc03b181 [id=930cfb0a66d5493ca68b5cc842bf2b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.395411 11554 tablet_bootstrap.cc:492] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:54.395831 11554 ts_tablet_manager.cc:1403] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.397s user 0.009s sys 0.030s
I20260809 01:45:54.396427 11113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b976164fd514af28ac8f97d26d83651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.396777 11110 tablet_service.cc:1511] Processing CreateTablet for tablet e0fdf4f4b1d44979936b37886bc60836 (DEFAULT_TABLE table=default.Table_8dde73a0e7f44af28232bbb7a0d57897 [id=cf06c89e274a4b81b578c53aaf7cd80a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.397959 11110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0fdf4f4b1d44979936b37886bc60836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.398293 11554 raft_consensus.cc:359] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.398907 11554 raft_consensus.cc:385] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:54.399140 11554 raft_consensus.cc:740] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:54.399741 11554 consensus_queue.cc:260] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.401836 11554 ts_tablet_manager.cc:1434] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20260809 01:45:54.402683 11554 tablet_bootstrap.cc:492] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:54.403151 12778 leader_election.cc:290] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:54.405505 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3684fa56f9549f492a96a5c765eda78" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:54.407245 11062 leader_election.cc:343] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:54.409039 11111 tablet_service.cc:1511] Processing CreateTablet for tablet d8e45cd77b334bdf9948c20ff3ff764c (DEFAULT_TABLE table=default.Table_a5ce279641de467da9670256492f78e2 [id=13eb3e4fa11e4dd493b69b01f07deafc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.410207 11111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e45cd77b334bdf9948c20ff3ff764c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.411782 11554 tablet_bootstrap.cc:654] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:54.417083 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 7c3f7eefb1f649d8a34f2c5de71d361e (DEFAULT_TABLE table=default.Table_65902a8894274e2ca34e86aa908b12f4 [id=b4d4214e9b9942ec87c7f2aaea9991ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.418072 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c3f7eefb1f649d8a34f2c5de71d361e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.431456 11115 tablet_service.cc:1511] Processing CreateTablet for tablet 62405c58875b433da4fb2d1cbcae1ab4 (DEFAULT_TABLE table=default.Table_55937d2e689c4822b6d1c366784f19cf [id=7408c688fcd5447d802070b8412e7837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.432559 11115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62405c58875b433da4fb2d1cbcae1ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.221352 10908 raft_consensus.cc:2468] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 2.
I20260809 01:45:54.230446 10877 tablet_service.cc:1511] Processing CreateTablet for tablet 6134c0e8fa79403b8785dda379c01a0f (DEFAULT_TABLE table=default.Table_0ccb4fdb37aa49a2bf24e84e07a54081 [id=7d3758d6271943b3b7b78eed034ec422]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.261195 12826 raft_consensus.cc:493] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:54.261452 10852 tablet_service.cc:1511] Processing CreateTablet for tablet 7eb2acab69734479aa729c65666b9081 (DEFAULT_TABLE table=default.Table_9e3630f9d80845d5bc57f0dde0550a41 [id=c250aefe7f6c49888699e646b31eae39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.262903 10847 tablet_service.cc:1511] Processing CreateTablet for tablet d2baec6d21e24c458bfbc2824408b5c4 (DEFAULT_TABLE table=default.Table_cfd1f608be9041d185710895f202f65e [id=d41a174cc20a4a6db3f8ae3b8622f48e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.264557 10854 tablet_service.cc:1511] Processing CreateTablet for tablet 74d6b03590c8472db8977bee5962aa6c (DEFAULT_TABLE table=default.Table_eb6df7e248d44109898ab900441bda8b [id=7701d6a8eff047ddac4f8775385c57ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.435063 11289 leader_election.cc:304] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7
I20260809 01:45:54.265687 10866 tablet_service.cc:1511] Processing CreateTablet for tablet fd8c5a45dec34006bd9c9894d336815c (DEFAULT_TABLE table=default.Table_99228426aa9e4d1b84a5f23af6de8669 [id=9b698cf563244d5c87b04054906bde65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.436270 11743 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 01:45:54.438612 11743 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:45:54.439292 11743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:45:54.436497 10847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2baec6d21e24c458bfbc2824408b5c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.445883 10854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74d6b03590c8472db8977bee5962aa6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.447572 12653 raft_consensus.cc:2804] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 01:45:54.449134 12653 raft_consensus.cc:697] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [term 2 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:54.449868 10852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eb2acab69734479aa729c65666b9081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.452665 12653 consensus_queue.cc:237] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.453913 10866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd8c5a45dec34006bd9c9894d336815c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.455856 12881 tablet.cc:1724] T cef88ffe9e95437da81793795742def3 P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:54.457844 10877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6134c0e8fa79403b8785dda379c01a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.461755 12826 raft_consensus.cc:515] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.464778 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0354e4c7bf354f22a9c3c389e0e8ec92" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:54.480846 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 1c5897b881c448dab0cafe9238fa4bbb (DEFAULT_TABLE table=default.Table_b254da32a96842368d87fa797c32915a [id=b098ff5e09a0421ca14522917ab6e4f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.482053 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c5897b881c448dab0cafe9238fa4bbb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:54.269980 10804 leader_election.cc:343] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:54.294445 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f022e6a57a4ec0a5a8c33159e97a8e" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:54.486135 10910 raft_consensus.cc:2468] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:54.487968 10804 leader_election.cc:304] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:54.489250 12826 leader_election.cc:290] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:54.490271 10595 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet a85e82d85424445d94b30402d41c53dc
I20260809 01:45:54.491633 10595 catalog_manager.cc:6807] ReplaceTablet: tablet a85e82d85424445d94b30402d41c53dc deleted and replaced by tablet 23f97a412594405bb5dca58e58f5dbc2
I20260809 01:45:54.492306 10588 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 9fffa48ccf58476da79dccd9c008b67c
I20260809 01:45:54.493690 10588 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 395) took 1035 ms. Trace:
I20260809 01:45:54.493974 10588 rpcz_store.cc:276] 0809 01:45:53.458665 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:53.458824 (+ 159us) service_pool.cc:224] Handling call
0809 01:45:53.460895 (+ 2071us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c86954d6 00007fa2c624db34 00007fa2c624dac2 00007fa2c624d9f1 00007fa2c624d96c 00007fa2c624cc22 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:53.462017 (+ 1122us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:53.462692 (+ 675us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:53.462814 (+ 122us) catalog_manager.cc:2850] Locking tablets
0809 01:45:53.984046 (+521232us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:54.491961 (+507915us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:54.492153 (+ 192us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:54.492231 (+ 78us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:54.493629 (+ 1398us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:54.301161 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:54.301614 (+ 453us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:54.301639 (+ 25us) write_op.cc:435] Acquired schema lock
0809 01:45:54.301650 (+ 11us) tablet.cc:662] Decoding operations
0809 01:45:54.301997 (+ 347us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:54.302034 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:54.302061 (+ 27us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:54.302386 (+ 325us) tablet.cc:701] Row locks acquired
0809 01:45:54.302404 (+ 18us) write_op.cc:260] PREPARE: finished
0809 01:45:54.302518 (+ 114us) write_op.cc:270] Start()
0809 01:45:54.302609 (+ 91us) write_op.cc:276] Timestamp: P: 1786239954302455 usec, L: 0
0809 01:45:54.302631 (+ 22us) op_driver.cc:348] REPLICATION: starting
0809 01:45:54.303313 (+ 682us) log.cc:844] Serialized 725 byte log entry
0809 01:45:54.487204 (+183891us) op_driver.cc:464] REPLICATION: finished
0809 01:45:54.490244 (+ 3040us) write_op.cc:301] APPLY: starting
0809 01:45:54.490290 (+ 46us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:54.490387 (+ 97us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:54.490475 (+ 88us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:54.490763 (+ 288us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:54.490925 (+ 162us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:54.490964 (+ 39us) write_op.cc:312] APPLY: finished
0809 01:45:54.491117 (+ 153us) log.cc:844] Serialized 29 byte log entry
0809 01:45:54.491689 (+ 572us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:54.491750 (+ 61us) write_op.cc:454] Released schema lock
0809 01:45:54.491780 (+ 30us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":3072384,"child_traces":[["op",{"apply.queue_time_us":2922,"num_ops":2,"prepare.queue_time_us":311405,"prepare.run_cpu_time_us":2503,"prepare.run_wall_time_us":4460,"raft.queue_time_us":96,"raft.run_cpu_time_us":568,"raft.run_wall_time_us":568,"replication_time_us":184494}]]}
I20260809 01:45:54.497822 12826 raft_consensus.cc:493] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:54.497350 10588 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_41c3d9aefb8b49a1afe6c6481fd8fd2f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:54.365899 12728 raft_consensus.cc:493] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:54.499737 12728 raft_consensus.cc:515] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:54.500643 10588 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_41c3d9aefb8b49a1afe6c6481fd8fd2f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:54.487468 11063 leader_election.cc:304] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:54.349082 11551 raft_consensus.cc:359] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.502058 11551 raft_consensus.cc:385] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:54.502173 12728 leader_election.cc:290] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:54.502404 11551 raft_consensus.cc:740] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:54.502609 12728 raft_consensus.cc:2749] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:54.408607 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3684fa56f9549f492a96a5c765eda78" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:54.502804 12778 raft_consensus.cc:2804] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:54.503154 12778 raft_consensus.cc:493] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:54.503243 10905 raft_consensus.cc:2468] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:54.503551 12778 raft_consensus.cc:3060] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:54.503129 11551 consensus_queue.cc:260] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.505786 11063 leader_election.cc:304] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:54.509763 12826 raft_consensus.cc:515] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.510572 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet a85e82d85424445d94b30402d41c53dc with delete_type TABLET_DATA_DELETED (replaced by tablet 23f97a412594405bb5dca58e58f5dbc2) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:54.510936 11337 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fffa48ccf58476da79dccd9c008b67c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:53 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:54.511093 12599 raft_consensus.cc:2804] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:54.511417 12599 raft_consensus.cc:493] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:54.511740 12599 raft_consensus.cc:3060] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:54.516327 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet a85e82d85424445d94b30402d41c53dc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a85e82d85424445d94b30402d41c53dc already in progress: creating tablet
W20260809 01:45:54.518044 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 9fffa48ccf58476da79dccd9c008b67c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9fffa48ccf58476da79dccd9c008b67c already in progress: creating tablet
I20260809 01:45:54.499118 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc9084f4ceb479d85dbc9146bb43c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.526728 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_ba11ce5b2eda44f897a3689070a56b40" } new_table_name: "default.Table_ddfd5630624142bc91b821055881a2ed" modify_external_catalogs: true
I20260809 01:45:54.531174 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 6542a2b55ce94835a31c732552d87222 (DEFAULT_TABLE table=default.Table_ae59ae0a1b7d4312856fd310b0c6c8b7 [id=96595f8bb6b84744bf7fc74f074a51ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.531910 10596 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet b6159d660d544b3e907a08372027c0e2 (table default.Table_99bad24613e94b6f840f637bea4978fc [id=ddfe680283854b9484dc3e9d6e626ab9]). Expected version 1 got 0
I20260809 01:45:54.549890 10596 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet cfb9afd3be194830ba99f5e20fdaca0a (table default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]). Expected version 1 got 0
I20260809 01:45:54.549907 12728 raft_consensus.cc:493] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:54.550590 12728 raft_consensus.cc:515] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.558193 12728 leader_election.cc:290] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:54.365402 11553 tablet_bootstrap.cc:492] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:54.567368 11125 tablet_service.cc:1511] Processing CreateTablet for tablet bc3e7e481fcb484bbeedf3e48d6b02c1 (DEFAULT_TABLE table=default.Table_197551df65a34e48be825222e02fc1cc [id=ec37d1dc50894321b6efecde89663c5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.567855 11553 ts_tablet_manager.cc:1403] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.729s user 0.008s sys 0.003s
I20260809 01:45:54.567905 10596 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 75e4315b3c644846ac572883c51f23de (table default.Table_ee7c1665ed2344278d1ee031c67438a4 [id=850f362a7f9a4531a3d7d6de29b695ba]). Expected version 4 got 0
I20260809 01:45:54.568763 11125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc3e7e481fcb484bbeedf3e48d6b02c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.569435 10596 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet cef88ffe9e95437da81793795742def3 (table default.Table_c54e4ecef7c145cc970af6169c5a9326 [id=40cb4403a8a94bd3938983441e9254d4]). Expected version 1 got 0
W20260809 01:45:54.537698 10581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.125:42063, user_credentials={real_user=slave}} blocked reactor thread for 353509us
I20260809 01:45:54.575094 12826 leader_election.cc:290] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:54.575194 11111 tablet_service.cc:1511] Processing CreateTablet for tablet e943a4fa04e542e5bcd82ee2ce022642 (DEFAULT_TABLE table=default.Table_c770b2448fca4145918eed5aa2fbc2f6 [id=24334f346e0a4bb89fd00d9758f71045]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.576851 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6542a2b55ce94835a31c732552d87222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.576956 11111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e943a4fa04e542e5bcd82ee2ce022642. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:54.581194 10804 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 314844us
I20260809 01:45:54.581768 11551 ts_tablet_manager.cc:1434] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.312s user 0.008s sys 0.002s
I20260809 01:45:54.591058 11105 tablet_service.cc:1511] Processing CreateTablet for tablet f44b03d0f5b4480ebf4a7f1d874820c7 (DEFAULT_TABLE table=default.Table_9f368169db0b40ee91c03bdc0b8d4785 [id=68875f8f88624a999bd31162c3300b0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.592221 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f44b03d0f5b4480ebf4a7f1d874820c7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:54.602489 10804 leader_election.cc:343] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:54.604190 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 5baead417d8f41299570b54677d4e173 (DEFAULT_TABLE table=default.Table_0a995cd327c14182b69fd09bec3096d6 [id=efef7087f29641e2a075e4da80015c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.605285 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5baead417d8f41299570b54677d4e173. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:54.609858 11063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 104139us
I20260809 01:45:54.617553 12599 raft_consensus.cc:515] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.629489 10804 leader_election.cc:304] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:54.558199 10861 consensus_meta.cc:287] T 4b976164fd514af28ac8f97d26d83651 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent flushing consensus metadata: real 0.506s user 0.008s sys 0.000s
I20260809 01:45:54.634850 12885 raft_consensus.cc:2749] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:54.621120 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 46bbf262a261468da91230a3ca68cbea (DEFAULT_TABLE table=default.Table_cc3003c3373b4f6cbcae8eb85030466f [id=b33ab68357c0431ca78bdc90bda6a7d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.646025 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46bbf262a261468da91230a3ca68cbea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.647523 10855 tablet_service.cc:1511] Processing CreateTablet for tablet f7d7a8f179244b3b9c9ca042af67e4c0 (DEFAULT_TABLE table=default.Table_836b8d3fc2e34cb786cadfc1794cb1ad [id=7b4191ad61c143eb8badf6c4f983ecee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.648723 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7d7a8f179244b3b9c9ca042af67e4c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.653260 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0354e4c7bf354f22a9c3c389e0e8ec92" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:54.658952 12599 leader_election.cc:290] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:54.607899 10805 leader_election.cc:343] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:54.669724 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 61856us
I20260809 01:45:54.604911 11553 raft_consensus.cc:359] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.623456 11118 tablet_service.cc:1511] Processing CreateTablet for tablet 6134c0e8fa79403b8785dda379c01a0f (DEFAULT_TABLE table=default.Table_0ccb4fdb37aa49a2bf24e84e07a54081 [id=7d3758d6271943b3b7b78eed034ec422]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.680755 11553 raft_consensus.cc:385] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:54.681123 11553 raft_consensus.cc:740] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:54.681774 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6134c0e8fa79403b8785dda379c01a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.681969 11553 consensus_queue.cc:260] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.684415 11553 ts_tablet_manager.cc:1434] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.116s user 0.006s sys 0.000s
I20260809 01:45:54.625331 11117 tablet_service.cc:1511] Processing CreateTablet for tablet d2baec6d21e24c458bfbc2824408b5c4 (DEFAULT_TABLE table=default.Table_cfd1f608be9041d185710895f202f65e [id=d41a174cc20a4a6db3f8ae3b8622f48e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.686368 11117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2baec6d21e24c458bfbc2824408b5c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.687316 11120 tablet_service.cc:1511] Processing CreateTablet for tablet 5e9afec0618e4fa293f6dad6769ae49c (DEFAULT_TABLE table=default.Table_1391d3ee4c884a43b3fa55ce4cf9d7d4 [id=0b559c6e6c364d2ab92c981378dc2719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.688768 11120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e9afec0618e4fa293f6dad6769ae49c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.626435 11116 tablet_service.cc:1511] Processing CreateTablet for tablet 7eb2acab69734479aa729c65666b9081 (DEFAULT_TABLE table=default.Table_9e3630f9d80845d5bc57f0dde0550a41 [id=c250aefe7f6c49888699e646b31eae39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.626981 11126 tablet_service.cc:1511] Processing CreateTablet for tablet 74d6b03590c8472db8977bee5962aa6c (DEFAULT_TABLE table=default.Table_eb6df7e248d44109898ab900441bda8b [id=7701d6a8eff047ddac4f8775385c57ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.627880 11107 tablet_service.cc:1511] Processing CreateTablet for tablet fd8c5a45dec34006bd9c9894d336815c (DEFAULT_TABLE table=default.Table_99228426aa9e4d1b84a5f23af6de8669 [id=9b698cf563244d5c87b04054906bde65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.628134 11112 tablet_service.cc:1511] Processing CreateTablet for tablet f7d7a8f179244b3b9c9ca042af67e4c0 (DEFAULT_TABLE table=default.Table_836b8d3fc2e34cb786cadfc1794cb1ad [id=7b4191ad61c143eb8badf6c4f983ecee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.629716 11106 tablet_service.cc:1511] Processing CreateTablet for tablet cb13f4c7b07348cba67b6b43a9b154bd (DEFAULT_TABLE table=default.Table_ce7493c5349e41d890087f1336689d4a [id=393bf4cab7eb48859853968aeae87b42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.691512 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74d6b03590c8472db8977bee5962aa6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.695001 11112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7d7a8f179244b3b9c9ca042af67e4c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.698590 11106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb13f4c7b07348cba67b6b43a9b154bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.702636 11107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd8c5a45dec34006bd9c9894d336815c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.703392 11119 tablet_service.cc:1558] Processing DeleteTablet for tablet 2790e2266ade4c698741a940ed49757e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:53 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:54.630012 11123 tablet_service.cc:1511] Processing CreateTablet for tablet f4985b1715f64adaae432d46dee4ea38 (DEFAULT_TABLE table=default.Table_8060ad8eb1be43f4a3c2e9dac705c0e3 [id=d3b5229e374c4e6bbce3104b99c494d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.636258 11125 tablet_service.cc:1511] Processing CreateTablet for tablet acc9084f4ceb479d85dbc9146bb43c8f (DEFAULT_TABLE table=default.Table_b2acb056eb544b64b517a5b0fdd3f131 [id=83da413be8d14f2e819cd6f2ca0befe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.659003 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63c1a610663a4a2fa58f6cdc5f8d86ce" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:54.663213 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2692a931500443d298e25a2b362b22a1" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:54.668246 11114 tablet_service.cc:1511] Processing CreateTablet for tablet 878b7a9d92df4824902b2b15295f65df (DEFAULT_TABLE table=default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:54.705480 10576 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet e5af40994c204c4f82e36a53e00f2dfe (table default.Table_15c9a494eb954149a4095ebfca7505a6 [id=e995a84f2635461ca81f05d2ece827c6], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e5af40994c204c4f82e36a53e00f2dfe consensus state has no leader
I20260809 01:45:54.672399 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9493892c1ff48e7a0558ff0fcb7a0f3" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:54.708138 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eb2acab69734479aa729c65666b9081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.732321 11129 tablet_service.cc:1511] Processing CreateTablet for tablet 1b25a1da87fb445e8bcf9468862ae976 (DEFAULT_TABLE table=default.Table_779631fdff574e96846c0dd384b81c9f [id=b289c25942334fd8946e86111451d9d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.743472 11124 tablet_service.cc:1558] Processing DeleteTablet for tablet 4aa1429f21024739b06bec88c3cb24d9 with delete_type TABLET_DATA_DELETED (replaced by tablet bdde640ea8c04e8381c71acae7ac809f) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:54.757896 11114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 878b7a9d92df4824902b2b15295f65df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.762465 11129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b25a1da87fb445e8bcf9468862ae976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.770363 11123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4985b1715f64adaae432d46dee4ea38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.770692 11118 tablet_service.cc:1511] Processing CreateTablet for tablet 6542a2b55ce94835a31c732552d87222 (DEFAULT_TABLE table=default.Table_ae59ae0a1b7d4312856fd310b0c6c8b7 [id=96595f8bb6b84744bf7fc74f074a51ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.771804 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6542a2b55ce94835a31c732552d87222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.774173 11125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc9084f4ceb479d85dbc9146bb43c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.784309 10595 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 75e4315b3c644846ac572883c51f23de (table default.Table_ee7c1665ed2344278d1ee031c67438a4 [id=850f362a7f9a4531a3d7d6de29b695ba]). Expected version 4 got 0
I20260809 01:45:54.784955 10595 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet cef88ffe9e95437da81793795742def3 (table default.Table_c54e4ecef7c145cc970af6169c5a9326 [id=40cb4403a8a94bd3938983441e9254d4]). Expected version 1 got 0
I20260809 01:45:54.785471 10595 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]). Expected version 3 got 0
I20260809 01:45:54.789714 10595 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet cfb9afd3be194830ba99f5e20fdaca0a (table default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]). Expected version 1 got 0
I20260809 01:45:54.661181 10864 tablet_service.cc:1511] Processing CreateTablet for tablet cb13f4c7b07348cba67b6b43a9b154bd (DEFAULT_TABLE table=default.Table_ce7493c5349e41d890087f1336689d4a [id=393bf4cab7eb48859853968aeae87b42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.621668 12778 raft_consensus.cc:515] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.811113 10595 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet b6159d660d544b3e907a08372027c0e2 (table default.Table_99bad24613e94b6f840f637bea4978fc [id=ddfe680283854b9484dc3e9d6e626ab9]). Expected version 1 got 0
I20260809 01:45:54.812420 10595 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet e3684fa56f9549f492a96a5c765eda78 (table default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]). Expected version 1 got 0
I20260809 01:45:54.814874 10592 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b (table default.Table_0bcdcc37d5f04f40ab4dd02b66baf4a6 [id=97ccdb35be6a437288222feb89f9e751]). Expected version 1 got 0
I20260809 01:45:54.815423 10592 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 75e4315b3c644846ac572883c51f23de (table default.Table_ee7c1665ed2344278d1ee031c67438a4 [id=850f362a7f9a4531a3d7d6de29b695ba]). Expected version 4 got 0
I20260809 01:45:54.816154 10592 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 3938bfd21b294fa6a25f0d8efc1ca29a (table default.Table_67bd4213ff9c478b9a51efdfe77bcdb7 [id=01721069a381465397d92855a7accc23]). Expected version 2 got 0
I20260809 01:45:54.817498 10598 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_c1a0ae6f926d45f780cc9f26ee4a1d49 [id=a178ba9920604ce988ec5c4fca1bec93] alter complete (version 1)
I20260809 01:45:54.744174 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3684fa56f9549f492a96a5c765eda78" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:54.819253 10910 raft_consensus.cc:3060] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:54.820335 10597 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.67:40463 (request call id 56) took 1717 ms (client timeout 1000 ms). Trace:
W20260809 01:45:54.820611 10597 rpcz_store.cc:269] 0809 01:45:53.102497 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:53.102591 (+ 94us) service_pool.cc:224] Handling call
0809 01:45:54.820258 (+1717667us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":4}
I20260809 01:45:54.746274 10861 tablet_service.cc:1511] Processing CreateTablet for tablet f4985b1715f64adaae432d46dee4ea38 (DEFAULT_TABLE table=default.Table_8060ad8eb1be43f4a3c2e9dac705c0e3 [id=d3b5229e374c4e6bbce3104b99c494d1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:54.822221 10589 catalog_manager.cc:5911] Failed to send AlterTable RPC for tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b (table default.Table_0bcdcc37d5f04f40ab4dd02b66baf4a6 [id=97ccdb35be6a437288222feb89f9e751], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b consensus state has no leader
W20260809 01:45:54.823388 10589 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 53) took 1723 ms (client timeout 1000 ms). Trace:
W20260809 01:45:54.823675 10589 rpcz_store.cc:269] 0809 01:45:53.100226 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:53.103885 (+ 3659us) service_pool.cc:224] Handling call
0809 01:45:54.823328 (+1719443us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":3}
I20260809 01:45:54.827188 10910 raft_consensus.cc:2468] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
W20260809 01:45:54.828105 10598 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.66:57751 (request call id 61) took 1817 ms (client timeout 1000 ms). Trace:
I20260809 01:45:54.685482 11553 tablet_bootstrap.cc:492] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:54.830440 12599 raft_consensus.cc:493] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:54.830960 12599 raft_consensus.cc:515] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.748651 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 878b7a9d92df4824902b2b15295f65df (DEFAULT_TABLE table=default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.751574 10874 tablet_service.cc:1558] Processing DeleteTablet for tablet 2790e2266ade4c698741a940ed49757e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:53 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:54.764489 10859 tablet_service.cc:1511] Processing CreateTablet for tablet 5e9afec0618e4fa293f6dad6769ae49c (DEFAULT_TABLE table=default.Table_1391d3ee4c884a43b3fa55ce4cf9d7d4 [id=0b559c6e6c364d2ab92c981378dc2719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.766467 10860 tablet_service.cc:1558] Processing DeleteTablet for tablet 4aa1429f21024739b06bec88c3cb24d9 with delete_type TABLET_DATA_DELETED (replaced by tablet bdde640ea8c04e8381c71acae7ac809f) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:54.768613 10867 tablet_service.cc:1511] Processing CreateTablet for tablet 1b25a1da87fb445e8bcf9468862ae976 (DEFAULT_TABLE table=default.Table_779631fdff574e96846c0dd384b81c9f [id=b289c25942334fd8946e86111451d9d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.770758 10871 tablet_service.cc:1511] Processing CreateTablet for tablet c7f9dd670ee146bea3cedffea82a91cc (DEFAULT_TABLE table=default.Table_51519b5a61474dba91f80b33789a2849 [id=aa77285868874c4db4fea36ac70035b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.775717 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 6542a2b55ce94835a31c732552d87222 (DEFAULT_TABLE table=default.Table_ae59ae0a1b7d4312856fd310b0c6c8b7 [id=96595f8bb6b84744bf7fc74f074a51ba]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:54.789700 10856 consensus_meta.cc:287] T bc3e7e481fcb484bbeedf3e48d6b02c1 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent flushing consensus metadata: real 0.581s user 0.005s sys 0.003s
I20260809 01:45:54.794555 10864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb13f4c7b07348cba67b6b43a9b154bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.849388 10867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b25a1da87fb445e8bcf9468862ae976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.850348 10859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e9afec0618e4fa293f6dad6769ae49c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.850867 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4985b1715f64adaae432d46dee4ea38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.853754 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f9dd670ee146bea3cedffea82a91cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.857880 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6542a2b55ce94835a31c732552d87222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.861044 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 878b7a9d92df4824902b2b15295f65df. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:54.829505 10598 rpcz_store.cc:269] 0809 01:45:53.010976 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:53.011138 (+ 162us) service_pool.cc:224] Handling call
0809 01:45:54.828048 (+1816910us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:53.282791 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:53.283783 (+ 992us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:53.283816 (+ 33us) write_op.cc:435] Acquired schema lock
0809 01:45:53.283859 (+ 43us) tablet.cc:662] Decoding operations
0809 01:45:53.284482 (+ 623us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:53.284554 (+ 72us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:53.285916 (+ 1362us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:53.286268 (+ 352us) tablet.cc:701] Row locks acquired
0809 01:45:53.286306 (+ 38us) write_op.cc:260] PREPARE: finished
0809 01:45:53.286545 (+ 239us) write_op.cc:270] Start()
0809 01:45:53.286666 (+ 121us) write_op.cc:276] Timestamp: P: 1786239953286396 usec, L: 0
0809 01:45:53.286709 (+ 43us) op_driver.cc:348] REPLICATION: starting
0809 01:45:53.287744 (+ 1035us) log.cc:844] Serialized 527 byte log entry
0809 01:45:53.371055 (+ 83311us) op_driver.cc:464] REPLICATION: finished
0809 01:45:53.372658 (+ 1603us) write_op.cc:301] APPLY: starting
0809 01:45:53.372731 (+ 73us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:53.372903 (+ 172us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:53.373107 (+ 204us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:53.373396 (+ 289us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:53.373692 (+ 296us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:53.373792 (+ 100us) write_op.cc:312] APPLY: finished
0809 01:45:53.374154 (+ 362us) log.cc:844] Serialized 23 byte log entry
0809 01:45:53.374395 (+ 241us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:53.374475 (+ 80us) write_op.cc:454] Released schema lock
0809 01:45:53.374557 (+ 82us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:53.374922 (+ 365us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:53.374968 (+ 46us) write_op.cc:454] Released schema lock
Related trace 'op':
0809 01:45:54.120528 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:54.120956 (+ 428us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:54.120982 (+ 26us) write_op.cc:435] Acquired schema lock
0809 01:45:54.120997 (+ 15us) tablet.cc:662] Decoding operations
0809 01:45:54.121269 (+ 272us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:54.121326 (+ 57us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:54.121351 (+ 25us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:54.121514 (+ 163us) tablet.cc:701] Row locks acquired
0809 01:45:54.121530 (+ 16us) write_op.cc:260] PREPARE: finished
0809 01:45:54.125963 (+ 4433us) write_op.cc:270] Start()
0809 01:45:54.126070 (+ 107us) write_op.cc:276] Timestamp: P: 1786239954125903 usec, L: 0
0809 01:45:54.126089 (+ 19us) op_driver.cc:348] REPLICATION: starting
0809 01:45:54.126572 (+ 483us) log.cc:844] Serialized 356 byte log entry
0809 01:45:54.382510 (+255938us) op_driver.cc:464] REPLICATION: finished
0809 01:45:54.382943 (+ 433us) write_op.cc:301] APPLY: starting
0809 01:45:54.383001 (+ 58us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:54.383092 (+ 91us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:54.383169 (+ 77us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:54.383337 (+ 168us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:54.383471 (+ 134us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:54.383524 (+ 53us) write_op.cc:312] APPLY: finished
0809 01:45:54.383694 (+ 170us) log.cc:844] Serialized 23 byte log entry
0809 01:45:54.384326 (+ 632us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:54.384385 (+ 59us) write_op.cc:454] Released schema lock
0809 01:45:54.384425 (+ 40us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:54.384758 (+ 333us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:54.384783 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4959,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":1383,"apply.run_cpu_time_us":2086,"apply.run_wall_time_us":2212,"num_ops":1,"prepare.queue_time_us":179893,"prepare.run_cpu_time_us":4192,"prepare.run_wall_time_us":45204,"raft.queue_time_us":14650,"raft.run_cpu_time_us":721,"raft.run_wall_time_us":722,"replication_time_us":84175}],["op",{"apply.queue_time_us":279,"apply.run_cpu_time_us":1219,"apply.run_wall_time_us":1777,"num_ops":1,"prepare.queue_time_us":143958,"prepare.run_cpu_time_us":2016,"prepare.run_wall_time_us":7100,"replication_time_us":256333}]]}
W20260809 01:45:54.791651 10865 consensus_meta.cc:287] T 46bbf262a261468da91230a3ca68cbea P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent flushing consensus metadata: real 0.594s user 0.003s sys 0.006s
I20260809 01:45:54.883846 11105 tablet_service.cc:1511] Processing CreateTablet for tablet 4cc185f76b12407fa0da8d6a96e7a06e (DEFAULT_TABLE table=default.Table_5f7c40d82c854aaa87deac85836396ca [id=e46576dcc09c48c08c346055f67cc701]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:54.891409 10592 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 54) took 766 ms (client timeout 1000 ms). Trace:
I20260809 01:45:54.891940 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 1c5897b881c448dab0cafe9238fa4bbb (DEFAULT_TABLE table=default.Table_b254da32a96842368d87fa797c32915a [id=b098ff5e09a0421ca14522917ab6e4f1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:54.891727 10592 rpcz_store.cc:269] 0809 01:45:54.125464 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:54.330219 (+204755us) service_pool.cc:224] Handling call
0809 01:45:54.371762 (+ 41543us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:54.891353 (+519591us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":5,"spinlock_wait_cycles":88490240}
I20260809 01:45:54.895732 10862 tablet_service.cc:1511] Processing CreateTablet for tablet 1c5897b881c448dab0cafe9238fa4bbb (DEFAULT_TABLE table=default.Table_b254da32a96842368d87fa797c32915a [id=b098ff5e09a0421ca14522917ab6e4f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.897585 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c5897b881c448dab0cafe9238fa4bbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.898223 10862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c5897b881c448dab0cafe9238fa4bbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.898634 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cc185f76b12407fa0da8d6a96e7a06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.908419 12599 leader_election.cc:290] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:54.829311 11063 leader_election.cc:304] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
W20260809 01:45:54.913802 11063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 84687.2us
I20260809 01:45:54.914144 11346 tablet_service.cc:1511] Processing CreateTablet for tablet d951db6def1148a89c19dac4890d4ab2 (DEFAULT_TABLE table=default.Table_50689eb6dda841ebbb61bfbe0e21c1cf [id=2139a1e3104345bba36e53e5988aa08a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.915205 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d951db6def1148a89c19dac4890d4ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.915877 11351 tablet_service.cc:1558] Processing DeleteTablet for tablet 60d356fcc48441ca80f2df29b533404f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:39 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:54.919203 12599 raft_consensus.cc:2804] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:54.920949 12599 raft_consensus.cc:697] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:54.923220 10856 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fffa48ccf58476da79dccd9c008b67c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:53 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:54.926276 10856 tablet_service.cc:1511] Processing CreateTablet for tablet 5baead417d8f41299570b54677d4e173 (DEFAULT_TABLE table=default.Table_0a995cd327c14182b69fd09bec3096d6 [id=efef7087f29641e2a075e4da80015c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.926980 12881 tablet.cc:1724] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:54.928153 10856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5baead417d8f41299570b54677d4e173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.924785 12599 consensus_queue.cc:237] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:54.935250 11124 tablet_service.cc:1511] Processing CreateTablet for tablet c7f9dd670ee146bea3cedffea82a91cc (DEFAULT_TABLE table=default.Table_51519b5a61474dba91f80b33789a2849 [id=aa77285868874c4db4fea36ac70035b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.936389 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f9dd670ee146bea3cedffea82a91cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:54.788472 11119 tablet_service.cc:1511] Processing CreateTablet for tablet 5baead417d8f41299570b54677d4e173 (DEFAULT_TABLE table=default.Table_0a995cd327c14182b69fd09bec3096d6 [id=efef7087f29641e2a075e4da80015c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:54.942487 11551 tablet_bootstrap.cc:492] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:54.947230 10865 tablet_service.cc:1558] Processing DeleteTablet for tablet a85e82d85424445d94b30402d41c53dc with delete_type TABLET_DATA_DELETED (replaced by tablet 23f97a412594405bb5dca58e58f5dbc2) from {username='slave'} at 127.0.0.1:45982
W20260809 01:45:54.954120 11063 leader_election.cc:343] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 7803a6aff02d42c1833f9b2cd7332b9f
I20260809 01:45:54.811136 11949 ts_tablet_manager.cc:1929] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:54.966275 11949 log.cc:1199] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/81da8d08131d4a3ba636ecfc7a7d2951
I20260809 01:45:54.967895 11949 ts_tablet_manager.cc:1950] T 81da8d08131d4a3ba636ecfc7a7d2951 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:54.943537 11119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5baead417d8f41299570b54677d4e173. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:54.983319 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 2790e2266ade4c698741a940ed49757e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2790e2266ade4c698741a940ed49757e already in progress: creating tablet
W20260809 01:45:54.984069 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 4aa1429f21024739b06bec88c3cb24d9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4aa1429f21024739b06bec88c3cb24d9 already in progress: creating tablet
I20260809 01:45:54.991504 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 261) took 6094 ms. Trace:
I20260809 01:45:55.003278 11949 rpcz_store.cc:276] 0809 01:45:48.897033 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:48.897152 (+ 119us) service_pool.cc:224] Handling call
0809 01:45:53.802483 (+4905331us) ts_tablet_manager.cc:1151] Deleting tablet 81da8d08131d4a3ba636ecfc7a7d2951
0809 01:45:53.802522 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:54.000222 (+197700us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:54.811029 (+810807us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:54.991444 (+180415us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4904764}
I20260809 01:45:54.986866 12653 raft_consensus.cc:493] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:55.016598 11384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63c1a610663a4a2fa58f6cdc5f8d86ce" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:55.019807 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet a85e82d85424445d94b30402d41c53dc with delete_type TABLET_DATA_DELETED (replaced by tablet 23f97a412594405bb5dca58e58f5dbc2) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:55.019873 11107 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fffa48ccf58476da79dccd9c008b67c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:53 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:55.020879 11116 tablet_service.cc:1511] Processing CreateTablet for tablet d951db6def1148a89c19dac4890d4ab2 (DEFAULT_TABLE table=default.Table_50689eb6dda841ebbb61bfbe0e21c1cf [id=2139a1e3104345bba36e53e5988aa08a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.021829 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d951db6def1148a89c19dac4890d4ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.022071 11110 tablet_service.cc:1558] Processing DeleteTablet for tablet 995a37a9e6b14889b876da5d76c89b84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:39 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:55.022774 11110 tablet_service.cc:1558] Processing DeleteTablet for tablet 197b5043f6d348f89853c073b256daa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:55.025288 12904 tablet_replica.cc:333] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:55.006556 12874 tablet_replica.cc:333] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
W20260809 01:45:55.026809 11113 consensus_meta.cc:287] T 4b976164fd514af28ac8f97d26d83651 P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 0.612s user 0.008s sys 0.000s
I20260809 01:45:55.027822 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7803a6aff02d42c1833f9b2cd7332b9f" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:55.028437 11389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9493892c1ff48e7a0558ff0fcb7a0f3" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:55.027335 12904 raft_consensus.cc:2243] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:55.029173 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3684fa56f9549f492a96a5c765eda78" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:55.030913 12904 raft_consensus.cc:2272] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:55.030220 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2692a931500443d298e25a2b362b22a1" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:55.033329 12904 ts_tablet_manager.cc:1916] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:55.034955 12778 leader_election.cc:290] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:55.021221 11107 tablet_service.cc:1511] Processing CreateTablet for tablet b93faf596f3c4dd286e600013c8db001 (DEFAULT_TABLE table=default.Table_79eb7d6b90da4bccbad721f2e840dd31 [id=87531d380b21454493fdffbcdc4ea825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.047339 11107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b93faf596f3c4dd286e600013c8db001. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:55.018929 10805 leader_election.cc:343] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:55.056795 11949 tablet_replica.cc:333] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
W20260809 01:45:55.056841 10805 leader_election.cc:343] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:55.057796 10805 leader_election.cc:343] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:55.045408 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f022e6a57a4ec0a5a8c33159e97a8e" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:55.058645 10905 raft_consensus.cc:3060] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:55.060403 12916 tablet.cc:1724] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:55.065471 10905 raft_consensus.cc:2468] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:55.067008 12889 raft_consensus.cc:493] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:55.072635 12889 raft_consensus.cc:515] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.076432 12889 leader_election.cc:290] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:55.079001 11063 leader_election.cc:304] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:55.090131 12778 raft_consensus.cc:2804] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:55.114396 12778 raft_consensus.cc:697] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:55.115935 12778 consensus_queue.cc:237] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:55.117345 11061 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.10.8.126:42157, user_credentials={real_user=slave}} blocked reactor thread for 134310us
W20260809 01:45:55.032423 11349 consensus_meta.cc:287] T 1b25a1da87fb445e8bcf9468862ae976 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.665s user 0.009s sys 0.000s
I20260809 01:45:55.135802 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7770aee1b7af415d995f63672fd05c01" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:55.137336 10805 leader_election.cc:343] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:55.147894 11554 tablet_bootstrap.cc:492] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:55.148392 11554 ts_tablet_manager.cc:1403] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.746s user 0.010s sys 0.121s
W20260809 01:45:55.152056 10858 consensus_meta.cc:287] T 4cc185f76b12407fa0da8d6a96e7a06e P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent flushing consensus metadata: real 0.563s user 0.004s sys 0.003s
W20260809 01:45:55.156911 11336 consensus_meta.cc:287] T 5e9afec0618e4fa293f6dad6769ae49c P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.733s user 0.008s sys 0.000s
I20260809 01:45:55.162019 12874 raft_consensus.cc:2243] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:55.164238 12874 raft_consensus.cc:2272] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:55.166625 12874 ts_tablet_manager.cc:1916] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:55.018529 12653 raft_consensus.cc:515] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.171448 12653 leader_election.cc:290] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:55.178874 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "995a37a9e6b14889b876da5d76c89b84" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:55.180461 11283 leader_election.cc:343] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: Tablet not RUNNING: STOPPED
W20260809 01:45:55.161439 11341 consensus_meta.cc:287] T 878b7a9d92df4824902b2b15295f65df P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.609s user 0.007s sys 0.000s
I20260809 01:45:55.152966 11554 raft_consensus.cc:359] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.186147 11554 raft_consensus.cc:385] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:55.186481 11554 raft_consensus.cc:740] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:55.189303 12889 raft_consensus.cc:493] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:55.187301 11554 consensus_queue.cc:260] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.189997 12889 raft_consensus.cc:515] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.191435 11551 tablet_bootstrap.cc:654] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:55.192119 12889 leader_election.cc:290] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:55.193539 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7a511512184f53921cea081f82aa37" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:55.194806 10805 leader_election.cc:343] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:55.194855 11062 leader_election.cc:343] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:55.195220 11062 leader_election.cc:304] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:55.196127 12599 raft_consensus.cc:2749] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:55.211088 11554 ts_tablet_manager.cc:1434] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.062s user 0.006s sys 0.000s
I20260809 01:45:55.212251 11554 tablet_bootstrap.cc:492] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
W20260809 01:45:55.229161 11062 leader_election.cc:343] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:55.234185 10847 consensus_meta.cc:287] T d2baec6d21e24c458bfbc2824408b5c4 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent flushing consensus metadata: real 0.664s user 0.007s sys 0.004s
I20260809 01:45:55.241487 10864 tablet_service.cc:1558] Processing DeleteTablet for tablet bd3662c307ba4ff78b8f3165b23acb1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:55.242022 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 81da8d08131d4a3ba636ecfc7a7d2951 (table default.Table_ce7493c5349e41d890087f1336689d4a [id=393bf4cab7eb48859853968aeae87b42]) successfully deleted
I20260809 01:45:55.246417 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f022e6a57a4ec0a5a8c33159e97a8e" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:55.248430 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7770aee1b7af415d995f63672fd05c01" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:55.251619 11062 leader_election.cc:343] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:55.250918 11289 leader_election.cc:343] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 995a37a9e6b14889b876da5d76c89b84
I20260809 01:45:55.254441 11289 leader_election.cc:304] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:55.263808 11949 raft_consensus.cc:2243] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:55.264297 11949 raft_consensus.cc:2272] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:55.266695 11949 ts_tablet_manager.cc:1916] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:55.286284 10847 tablet_service.cc:1511] Processing CreateTablet for tablet d951db6def1148a89c19dac4890d4ab2 (DEFAULT_TABLE table=default.Table_50689eb6dda841ebbb61bfbe0e21c1cf [id=2139a1e3104345bba36e53e5988aa08a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.288084 10847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d951db6def1148a89c19dac4890d4ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.298273 11551 tablet_bootstrap.cc:492] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:55.298722 11551 ts_tablet_manager.cc:1403] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.357s user 0.003s sys 0.014s
I20260809 01:45:55.302080 11551 raft_consensus.cc:359] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.302802 11551 raft_consensus.cc:385] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:55.303148 11551 raft_consensus.cc:740] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:55.303933 11551 consensus_queue.cc:260] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.313671 11553 tablet_bootstrap.cc:654] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:55.326577 10576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.126:42157, user_credentials={real_user=slave}} blocked reactor thread for 228503us
W20260809 01:45:55.329059 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet a85e82d85424445d94b30402d41c53dc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a85e82d85424445d94b30402d41c53dc already in progress: creating tablet
I20260809 01:45:55.330205 11554 tablet_bootstrap.cc:654] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:55.333668 10591 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_ed7186006163408da3c63d5ad3528807" } new_table_name: "default.Table_109c891eea264584b9e1ceab647f32aa" modify_external_catalogs: true
I20260809 01:45:55.335067 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_b4e6661838ac4bd7a5fb72595976f957"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:55.336550 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b4e6661838ac4bd7a5fb72595976f957 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:55.343362 10597 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_7f67458772ff440387609b94eba60aa4" } modify_external_catalogs: true
I20260809 01:45:55.343853 10597 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_7f67458772ff440387609b94eba60aa4" } modify_external_catalogs: true
I20260809 01:45:55.344959 10598 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_ee7c1665ed2344278d1ee031c67438a4" } modify_external_catalogs: true
I20260809 01:45:55.345409 10598 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_ee7c1665ed2344278d1ee031c67438a4" } modify_external_catalogs: true
W20260809 01:45:55.349783 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 9fffa48ccf58476da79dccd9c008b67c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9fffa48ccf58476da79dccd9c008b67c already in progress: creating tablet
I20260809 01:45:55.353602 11949 ts_tablet_manager.cc:1929] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:55.354087 11949 log.cc:1199] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/bb7a8b6d06064addb02979a7feedf756
I20260809 01:45:55.354712 11949 ts_tablet_manager.cc:1950] T bb7a8b6d06064addb02979a7feedf756 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:55.355458 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 262) took 6273 ms. Trace:
I20260809 01:45:55.355789 11949 rpcz_store.cc:276] 0809 01:45:49.082343 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:49.082504 (+ 161us) service_pool.cc:224] Handling call
0809 01:45:55.056577 (+5974073us) ts_tablet_manager.cc:1151] Deleting tablet bb7a8b6d06064addb02979a7feedf756
0809 01:45:55.056632 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:55.284432 (+227800us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:55.351463 (+ 67031us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:55.355403 (+ 3940us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":206110,"tablet-delete.queue_time_us":5973524}
I20260809 01:45:55.357144 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 263) took 5361 ms. Trace:
I20260809 01:45:55.357429 11949 rpcz_store.cc:276] 0809 01:45:49.995890 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:49.995990 (+ 100us) service_pool.cc:224] Handling call
0809 01:45:55.356826 (+5360836us) ts_tablet_manager.cc:1151] Deleting tablet 117d5fad26fd42dab19cfc1687364506
0809 01:45:55.356863 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:55.357097 (+ 234us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5360308}
I20260809 01:45:55.358556 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 265) took 5322 ms. Trace:
I20260809 01:45:55.358855 11949 rpcz_store.cc:276] 0809 01:45:50.036191 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:50.036323 (+ 132us) service_pool.cc:224] Handling call
0809 01:45:55.358233 (+5321910us) ts_tablet_manager.cc:1151] Deleting tablet 14cd02276f26405c95f028edf967bf10
0809 01:45:55.358279 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:55.358510 (+ 231us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5321289}
I20260809 01:45:55.360024 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 270) took 5081 ms. Trace:
I20260809 01:45:55.360324 11949 rpcz_store.cc:276] 0809 01:45:50.278639 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:50.279103 (+ 464us) service_pool.cc:224] Handling call
0809 01:45:55.359682 (+5080579us) ts_tablet_manager.cc:1151] Deleting tablet ebc209c696ac4c669961f5ed8392af40
0809 01:45:55.359737 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:55.359975 (+ 238us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5072755}
I20260809 01:45:55.361213 11949 tablet_replica.cc:333] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:55.364846 11554 tablet_bootstrap.cc:492] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:55.365588 11554 ts_tablet_manager.cc:1403] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.154s user 0.012s sys 0.005s
I20260809 01:45:55.357342 12874 ts_tablet_manager.cc:1929] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:55.372231 12874 log.cc:1199] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/60d356fcc48441ca80f2df29b533404f
I20260809 01:45:55.373153 11949 raft_consensus.cc:2243] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:55.373816 11949 raft_consensus.cc:2272] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:55.374989 12874 ts_tablet_manager.cc:1950] T 60d356fcc48441ca80f2df29b533404f P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:55.388290 11949 ts_tablet_manager.cc:1916] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:55.404572 10596 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.67:40463 (request call id 57) took 1233 ms (client timeout 1000 ms). Trace:
W20260809 01:45:55.407858 10596 rpcz_store.cc:269] 0809 01:45:54.170929 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:54.339333 (+168404us) service_pool.cc:224] Handling call
0809 01:45:54.367248 (+ 27915us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:55.404506 (+1037258us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":10,"spinlock_wait_cycles":58323712}
W20260809 01:45:55.419577 10595 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.66:57751 (request call id 62) took 973 ms (client timeout 1000 ms). Trace:
W20260809 01:45:55.419914 10595 rpcz_store.cc:269] 0809 01:45:54.446022 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:54.492115 (+ 46093us) service_pool.cc:224] Handling call
0809 01:45:55.419520 (+927405us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":361660,"reported_tablets":14,"spinlock_wait_cycles":8448}
I20260809 01:45:55.432643 11119 tablet_service.cc:1511] Processing CreateTablet for tablet b38a0a3033cc4328a0200e11c8eb5c31 (DEFAULT_TABLE table=default.Table_946a0fc83ebc47b99a4d35abd513a966 [id=cccf27c2588f40e79723a91fa276642c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.437983 11105 tablet_service.cc:1511] Processing CreateTablet for tablet c151a35902014f10902fb545d1e646e4 (DEFAULT_TABLE table=default.Table_57b8282c32f843b1882c94f5479bd35a [id=34d18887856a44c0a716874a85e3c49d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.461697 10858 tablet_service.cc:1558] Processing DeleteTablet for tablet 72b4850de7d740d794283ca93152fdc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:47 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:55.470427 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c151a35902014f10902fb545d1e646e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.442071 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 2126670a2f29408a9ba4b949f966bf76 (DEFAULT_TABLE table=default.Table_7399370c91a7495faa66e98683c688d0 [id=43255c746b5c4fcc91454fcad2c801a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.472368 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2126670a2f29408a9ba4b949f966bf76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.495175 11107 tablet_service.cc:1511] Processing CreateTablet for tablet 01e9f79113fb48c58e7fdbfab4fb49b8 (DEFAULT_TABLE table=default.Table_d5bd0ddd910f400b88147ea461999b39 [id=db57dff908bf4fc38861e80de59e00c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.502326 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7a511512184f53921cea081f82aa37" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:55.519073 11117 tablet_service.cc:1511] Processing CreateTablet for tablet d8ca63cedd3b4626949a0ec07bc26290 (DEFAULT_TABLE table=default.Table_199eacafb2d74a8190d24018b14db258 [id=e224e231c3e64f1ca6887a3593acf948]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.520272 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet b2726cb09f734ed18bb9379f94f3f22a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:41 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:55.521335 11106 tablet_service.cc:1558] Processing DeleteTablet for tablet c528b45de0d14c19af5d6836b7a14f1c with delete_type TABLET_DATA_DELETED (replaced by tablet 31874b7e70a847a4885bd09da40a6902) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:55.530695 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_13127c445be045458496dece32356891"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:55.532380 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13127c445be045458496dece32356891 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:55.505669 10804 leader_election.cc:343] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:55.541790 10804 leader_election.cc:304] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:55.542644 12889 raft_consensus.cc:2749] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:55.542989 10804 leader_election.cc:343] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:55.543485 10804 leader_election.cc:304] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:55.544327 12889 raft_consensus.cc:2749] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:55.544775 10804 leader_election.cc:343] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:55.581753 11949 ts_tablet_manager.cc:1929] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:55.582166 11949 log.cc:1199] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/a38bd344108649ed8e1687f82154000b
I20260809 01:45:55.582898 11949 ts_tablet_manager.cc:1950] T a38bd344108649ed8e1687f82154000b P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:55.583673 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 269) took 5305 ms. Trace:
I20260809 01:45:55.583981 11949 rpcz_store.cc:276] 0809 01:45:50.278310 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:50.286762 (+ 8452us) service_pool.cc:224] Handling call
0809 01:45:55.361072 (+5074310us) ts_tablet_manager.cc:1151] Deleting tablet a38bd344108649ed8e1687f82154000b
0809 01:45:55.361116 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:55.425569 (+ 64453us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:55.581629 (+156060us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:55.583615 (+ 1986us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5073782}
I20260809 01:45:55.585023 11949 tablet_replica.cc:333] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:55.586005 11107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01e9f79113fb48c58e7fdbfab4fb49b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.587558 11117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8ca63cedd3b4626949a0ec07bc26290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.589136 11119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b38a0a3033cc4328a0200e11c8eb5c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.613890 12904 ts_tablet_manager.cc:1929] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:55.614485 12904 log.cc:1199] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/995a37a9e6b14889b876da5d76c89b84
I20260809 01:45:55.615561 12904 ts_tablet_manager.cc:1950] T 995a37a9e6b14889b876da5d76c89b84 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:55.630703 11551 ts_tablet_manager.cc:1434] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.331s user 0.005s sys 0.000s
I20260809 01:45:55.631749 11551 tablet_bootstrap.cc:492] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:55.643967 12904 tablet_replica.cc:333] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:55.644855 12904 raft_consensus.cc:2243] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:55.645401 12904 raft_consensus.cc:2272] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:55.545249 10804 leader_election.cc:304] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:55.647490 12904 ts_tablet_manager.cc:1916] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:55.647791 11553 tablet_bootstrap.cc:492] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:55.648237 11553 ts_tablet_manager.cc:1403] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.963s user 0.006s sys 0.007s
I20260809 01:45:55.659636 11949 raft_consensus.cc:2243] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:55.659780 12889 raft_consensus.cc:2749] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:55.660121 11949 raft_consensus.cc:2272] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:55.660490 10804 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 115760us
W20260809 01:45:55.661053 10804 leader_election.cc:343] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:55.661711 11949 ts_tablet_manager.cc:1916] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:55.666563 12904 ts_tablet_manager.cc:1929] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:55.666913 12904 log.cc:1199] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/197b5043f6d348f89853c073b256daa4
I20260809 01:45:55.667557 12904 ts_tablet_manager.cc:1950] T 197b5043f6d348f89853c073b256daa4 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:55.670077 12904 tablet_replica.cc:333] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:55.661381 10804 leader_election.cc:304] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:55.681892 12889 raft_consensus.cc:2749] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:55.683038 10804 leader_election.cc:343] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:55.683454 10804 leader_election.cc:304] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:55.684108 12889 raft_consensus.cc:2749] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:55.686623 10804 leader_election.cc:343] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:55.687624 12904 raft_consensus.cc:2243] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:55.688191 12904 raft_consensus.cc:2272] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:55.695282 12904 ts_tablet_manager.cc:1916] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:55.695992 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 444868us
I20260809 01:45:55.696151 12889 raft_consensus.cc:493] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:55.721704 12904 ts_tablet_manager.cc:1929] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:55.722103 12904 log.cc:1199] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/b2726cb09f734ed18bb9379f94f3f22a
I20260809 01:45:55.722081 12889 raft_consensus.cc:515] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.724160 12889 leader_election.cc:290] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:55.687307 10804 leader_election.cc:304] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:55.726056 12889 raft_consensus.cc:2749] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:55.726495 11554 raft_consensus.cc:359] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.727391 11554 raft_consensus.cc:385] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:55.727661 11554 raft_consensus.cc:740] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:55.728367 11554 consensus_queue.cc:260] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:55.732906 10581 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 63c1a610663a4a2fa58f6cdc5f8d86ce (table default.Table_bd6eedffdf6440efa3e776b7079015d8 [id=8a2084683a7740db80e2c52d3f4c9506], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 63c1a610663a4a2fa58f6cdc5f8d86ce consensus state has no leader
I20260809 01:45:55.736267 12864 raft_consensus.cc:2749] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:55.736490 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 995a37a9e6b14889b876da5d76c89b84 (table default.Table_50f4db524f554499a2dab6d3bc48f053 [id=85ee30d1d77d47f691e6a0dd498d9bca]) successfully deleted
W20260809 01:45:55.737267 10586 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1b7c3dcab1594a87bd8d12e1dbfd8e3b has no consensus state
I20260809 01:45:55.737571 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 197b5043f6d348f89853c073b256daa4 (table default.Table_5f5f7b17133f4f4d9cfcac1aa0d270fa [id=12a77217eec442c6a9199a418d9d5bb8]) successfully deleted
I20260809 01:45:55.738013 12864 raft_consensus.cc:493] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:55.738050 10586 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:37956 (request call id 407) took 1667 ms. Trace:
I20260809 01:45:55.738493 12864 raft_consensus.cc:515] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.741144 10855 tablet_service.cc:1511] Processing CreateTablet for tablet b93faf596f3c4dd286e600013c8db001 (DEFAULT_TABLE table=default.Table_79eb7d6b90da4bccbad721f2e840dd31 [id=87531d380b21454493fdffbcdc4ea825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.742154 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b93faf596f3c4dd286e600013c8db001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.743553 12864 leader_election.cc:290] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:55.744549 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet bb7a8b6d06064addb02979a7feedf756 (table default.Table_7399370c91a7495faa66e98683c688d0 [id=43255c746b5c4fcc91454fcad2c801a6]) successfully deleted
I20260809 01:45:55.722919 12904 ts_tablet_manager.cc:1950] T b2726cb09f734ed18bb9379f94f3f22a P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:55.745934 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e03033672e947eab8cc143263281133" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:55.749284 11283 leader_election.cc:343] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: 197b5043f6d348f89853c073b256daa4
W20260809 01:45:55.745424 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 117d5fad26fd42dab19cfc1687364506 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 117d5fad26fd42dab19cfc1687364506 already in progress: creating tablet
W20260809 01:45:55.751252 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 14cd02276f26405c95f028edf967bf10 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14cd02276f26405c95f028edf967bf10 already in progress: creating tablet
I20260809 01:45:55.751658 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet b2726cb09f734ed18bb9379f94f3f22a (table default.Table_03cfb94f08614dc5bcdaf9eb182adc1b [id=d23f238a284a47f2be2faf340b853440]) successfully deleted
W20260809 01:45:55.752190 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet ebc209c696ac4c669961f5ed8392af40 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ebc209c696ac4c669961f5ed8392af40 already in progress: creating tablet
I20260809 01:45:55.752893 11551 tablet_bootstrap.cc:654] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:55.738487 10586 rpcz_store.cc:276] 0809 01:45:54.070174 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:54.441066 (+370892us) service_pool.cc:224] Handling call
0809 01:45:54.526596 (+ 85530us) spinlock_profiling.cc:243] Waited 44.9 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8695846 00007fa2c6252634 00007fa2c62525c2 00007fa2c62524f1 00007fa2c625246c 00007fa2c6251722 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:54.527321 (+ 725us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:54.527923 (+ 602us) catalog_manager.cc:3751] Apply alter schema
0809 01:45:54.528537 (+ 614us) catalog_manager.cc:3783] Acquired catalog manager lock
0809 01:45:54.528735 (+ 198us) catalog_manager.cc:3941] Updating metadata on disk
0809 01:45:55.736056 (+1207321us) catalog_manager.cc:3972] Committing alterations to in-memory state
0809 01:45:55.737926 (+ 1870us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:55.417882 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:55.430155 (+ 12273us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:55.430185 (+ 30us) write_op.cc:435] Acquired schema lock
0809 01:45:55.430195 (+ 10us) tablet.cc:662] Decoding operations
0809 01:45:55.431005 (+ 810us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:55.431046 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:55.431069 (+ 23us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:55.431229 (+ 160us) tablet.cc:701] Row locks acquired
0809 01:45:55.431245 (+ 16us) write_op.cc:260] PREPARE: finished
0809 01:45:55.431384 (+ 139us) write_op.cc:270] Start()
0809 01:45:55.431469 (+ 85us) write_op.cc:276] Timestamp: P: 1786239955431172 usec, L: 0
0809 01:45:55.431486 (+ 17us) op_driver.cc:348] REPLICATION: starting
0809 01:45:55.534041 (+102555us) log.cc:844] Serialized 405 byte log entry
0809 01:45:55.538054 (+ 4013us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678d951 00007fa2c6820208 00007fa2c6caff61 00007fa2c6caf738 00007fa2c6cb43d1 00007fa2c6cb4389 00007fa2c6cb4319 00007fa2c6cb42e1 00007fa2c6cb350d 00000000005bd8c4 00000000005bd6f9
0809 01:45:55.598848 (+ 60794us) op_driver.cc:464] REPLICATION: finished
0809 01:45:55.624759 (+ 25911us) write_op.cc:301] APPLY: starting
0809 01:45:55.624839 (+ 80us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:55.624937 (+ 98us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:55.624958 (+ 21us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:55.625133 (+ 175us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:55.625265 (+ 132us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:55.625311 (+ 46us) write_op.cc:312] APPLY: finished
0809 01:45:55.625491 (+ 180us) log.cc:844] Serialized 23 byte log entry
0809 01:45:55.625701 (+ 210us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:55.625768 (+ 67us) write_op.cc:454] Released schema lock
0809 01:45:55.625802 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:55.626052 (+ 250us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:55.626079 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":103346944,"child_traces":[["op",{"apply.queue_time_us":25745,"apply.run_cpu_time_us":1267,"apply.run_wall_time_us":1268,"num_ops":1,"prepare.queue_time_us":887728,"prepare.run_cpu_time_us":4698,"prepare.run_wall_time_us":120396,"raft.queue_time_us":3961,"raft.run_cpu_time_us":687,"raft.run_wall_time_us":687,"replication_time_us":167264,"spinlock_wait_cycles":234583424,"thread_start_us":25460,"threads_started":1}]]}
I20260809 01:45:55.754971 10846 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:45966 (request call id 48) took 1209 ms. Trace:
I20260809 01:45:55.757058 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet a38bd344108649ed8e1687f82154000b (table default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]) successfully deleted
I20260809 01:45:55.759620 10588 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:37956 (request call id 408) took 1584 ms. Trace:
I20260809 01:45:55.759830 12904 tablet_replica.cc:333] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:55.760771 12904 raft_consensus.cc:2243] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:55.761212 12904 raft_consensus.cc:2272] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:55.761538 11553 raft_consensus.cc:359] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.762194 11553 raft_consensus.cc:385] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:55.762514 11553 raft_consensus.cc:740] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:55.759930 10588 rpcz_store.cc:276] 0809 01:45:54.175441 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:54.497028 (+321587us) service_pool.cc:224] Handling call
0809 01:45:54.501005 (+ 3977us) catalog_manager.cc:2127] Acquired catalog manager lock
0809 01:45:54.533843 (+ 32838us) catalog_manager.cc:2173] Created new table and tablet info
0809 01:45:55.758471 (+1224628us) catalog_manager.cc:2233] Wrote table and tablets to system table
0809 01:45:55.758954 (+ 483us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0809 01:45:55.759438 (+ 484us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:55.538403 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:55.538844 (+ 441us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:55.538873 (+ 29us) write_op.cc:435] Acquired schema lock
0809 01:45:55.538887 (+ 14us) tablet.cc:662] Decoding operations
0809 01:45:55.539165 (+ 278us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:55.539205 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:55.539229 (+ 24us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:55.539589 (+ 360us) tablet.cc:701] Row locks acquired
0809 01:45:55.539607 (+ 18us) write_op.cc:260] PREPARE: finished
0809 01:45:55.539792 (+ 185us) write_op.cc:270] Start()
0809 01:45:55.539956 (+ 164us) write_op.cc:276] Timestamp: P: 1786239955539553 usec, L: 0
0809 01:45:55.539994 (+ 38us) op_driver.cc:348] REPLICATION: starting
0809 01:45:55.540931 (+ 937us) log.cc:844] Serialized 433 byte log entry
0809 01:45:55.624630 (+ 83699us) op_driver.cc:464] REPLICATION: finished
0809 01:45:55.626307 (+ 1677us) write_op.cc:301] APPLY: starting
0809 01:45:55.626361 (+ 54us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:55.626464 (+ 103us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:55.626551 (+ 87us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:55.626987 (+ 436us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:55.627132 (+ 145us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 01:45:55.627182 (+ 50us) write_op.cc:312] APPLY: finished
0809 01:45:55.627352 (+ 170us) log.cc:844] Serialized 29 byte log entry
0809 01:45:55.627503 (+ 151us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:55.627567 (+ 64us) write_op.cc:454] Released schema lock
0809 01:45:55.627601 (+ 34us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1551,"apply.run_cpu_time_us":1462,"apply.run_wall_time_us":1461,"num_ops":2,"prepare.queue_time_us":960122,"prepare.run_cpu_time_us":2857,"prepare.run_wall_time_us":4732,"replication_time_us":84547,"thread_start_us":16130,"threads_started":1}]]}
I20260809 01:45:55.763145 12904 ts_tablet_manager.cc:1916] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:55.765477 10599 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:37956 (request call id 405) took 1787 ms. Trace:
I20260809 01:45:55.755366 10846 rpcz_store.cc:276] 0809 01:45:54.545863 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:54.546047 (+ 184us) service_pool.cc:224] Handling call
0809 01:45:55.754890 (+1208843us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:55.765827 10599 rpcz_store.cc:276] 0809 01:45:53.978451 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:53.978566 (+ 115us) service_pool.cc:224] Handling call
0809 01:45:54.085330 (+106764us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c86951c6 00007fa2c6249034 00007fa2c6248fc2 00007fa2c6248ef1 00007fa2c6248e6c 00007fa2c6248122 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:54.087397 (+ 2067us) catalog_manager.cc:2127] Acquired catalog manager lock
0809 01:45:54.088830 (+ 1433us) catalog_manager.cc:2173] Created new table and tablet info
0809 01:45:55.764378 (+1675548us) catalog_manager.cc:2233] Wrote table and tablets to system table
0809 01:45:55.764840 (+ 462us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0809 01:45:55.765307 (+ 467us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:54.667228 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:54.738015 (+ 70787us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:54.738091 (+ 76us) write_op.cc:435] Acquired schema lock
0809 01:45:54.738104 (+ 13us) tablet.cc:662] Decoding operations
0809 01:45:54.738915 (+ 811us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:54.738994 (+ 79us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:54.739030 (+ 36us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:54.739622 (+ 592us) tablet.cc:701] Row locks acquired
0809 01:45:54.739677 (+ 55us) write_op.cc:260] PREPARE: finished
0809 01:45:54.739972 (+ 295us) write_op.cc:270] Start()
0809 01:45:54.740183 (+ 211us) write_op.cc:276] Timestamp: P: 1786239954739886 usec, L: 0
0809 01:45:54.740201 (+ 18us) op_driver.cc:348] REPLICATION: starting
0809 01:45:54.741272 (+ 1071us) log.cc:844] Serialized 433 byte log entry
0809 01:45:55.398298 (+657026us) op_driver.cc:464] REPLICATION: finished
0809 01:45:55.440539 (+ 42241us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5800051bd0. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c677e81c 00007fa2c6785133 00007fa2c67850b9 00007fa2c6785049 00007fa2c6785011 00007fa2c67841dd 00000000005bd8c4 00000000005bd6f9 00007fa2bf6bedb5 00007fa2bf6c1121
0809 01:45:55.524158 (+ 83619us) write_op.cc:301] APPLY: starting
0809 01:45:55.524274 (+ 116us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:55.524470 (+ 196us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:55.524519 (+ 49us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:55.525396 (+ 877us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:55.526015 (+ 619us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 01:45:55.526195 (+ 180us) write_op.cc:312] APPLY: finished
0809 01:45:55.526461 (+ 266us) log.cc:844] Serialized 29 byte log entry
0809 01:45:55.526716 (+ 255us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:55.526850 (+ 134us) write_op.cc:454] Released schema lock
0809 01:45:55.526883 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:55.629707 (+102824us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:55.629739 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":243393280,"child_traces":[["op",{"apply.queue_time_us":125570,"apply.run_cpu_time_us":2614,"apply.run_wall_time_us":105562,"num_ops":2,"prepare.queue_time_us":541329,"prepare.run_cpu_time_us":12592,"prepare.run_wall_time_us":750567,"raft.queue_time_us":80421,"raft.run_cpu_time_us":2523,"raft.run_wall_time_us":108343,"replication_time_us":657928,"spinlock_wait_cycles":246158080,"thread_start_us":183577,"threads_started":2}]]}
I20260809 01:45:55.787272 10589 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:37956 (request call id 411) took 1142 ms. Trace:
I20260809 01:45:55.787719 10589 rpcz_store.cc:276] 0809 01:45:54.644953 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:54.824280 (+179327us) service_pool.cc:224] Handling call
0809 01:45:55.334944 (+510664us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c86951c6 00007fa2c6249034 00007fa2c6248fc2 00007fa2c6248ef1 00007fa2c6248e6c 00007fa2c6248122 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:55.336915 (+ 1971us) catalog_manager.cc:2127] Acquired catalog manager lock
0809 01:45:55.401439 (+ 64524us) catalog_manager.cc:2173] Created new table and tablet info
0809 01:45:55.785860 (+384421us) catalog_manager.cc:2233] Wrote table and tablets to system table
0809 01:45:55.786413 (+ 553us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0809 01:45:55.787022 (+ 609us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:55.564685 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:55.565398 (+ 713us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:55.565430 (+ 32us) write_op.cc:435] Acquired schema lock
0809 01:45:55.565467 (+ 37us) tablet.cc:662] Decoding operations
0809 01:45:55.565970 (+ 503us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:55.566046 (+ 76us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:55.566093 (+ 47us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:55.566464 (+ 371us) tablet.cc:701] Row locks acquired
0809 01:45:55.566481 (+ 17us) write_op.cc:260] PREPARE: finished
0809 01:45:55.566674 (+ 193us) write_op.cc:270] Start()
0809 01:45:55.566816 (+ 142us) write_op.cc:276] Timestamp: P: 1786239955566439 usec, L: 0
0809 01:45:55.566857 (+ 41us) op_driver.cc:348] REPLICATION: starting
0809 01:45:55.567615 (+ 758us) log.cc:844] Serialized 433 byte log entry
0809 01:45:55.775426 (+207811us) op_driver.cc:464] REPLICATION: finished
0809 01:45:55.781116 (+ 5690us) write_op.cc:301] APPLY: starting
0809 01:45:55.781217 (+ 101us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:55.781383 (+ 166us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:55.781478 (+ 95us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:55.782066 (+ 588us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:55.782193 (+ 127us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 01:45:55.782231 (+ 38us) write_op.cc:312] APPLY: finished
0809 01:45:55.782441 (+ 210us) log.cc:844] Serialized 29 byte log entry
0809 01:45:55.783195 (+ 754us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:55.783377 (+ 182us) write_op.cc:454] Released schema lock
0809 01:45:55.783415 (+ 38us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":1161524608,"child_traces":[["op",{"apply.queue_time_us":5497,"num_ops":2,"prepare.queue_time_us":161910,"prepare.run_cpu_time_us":2538,"prepare.run_wall_time_us":25188,"replication_time_us":208367}]]}
I20260809 01:45:55.788549 10595 catalog_manager.cc:5719] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66). New cstate: current_term: 1 leader_uuid: "33b5652538e64ed58a09993d3188f8d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:55.794813 10589 master_service.cc:960] ReplaceTablet: received request to replace tablet efe4c7b15604442eb7be83fe19bb54f4 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:55.793061 10595 catalog_manager.cc:5719] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66). New cstate: current_term: 1 leader_uuid: "33b5652538e64ed58a09993d3188f8d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:55.795782 10595 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet e3684fa56f9549f492a96a5c765eda78 (table default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]). Expected version 1 got 0
W20260809 01:45:55.797899 10591 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3e03033672e947eab8cc143263281133 consensus state has no leader
I20260809 01:45:55.798476 10591 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:37956 (request call id 409) took 1622 ms. Trace:
I20260809 01:45:55.801839 10598 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 75e4315b3c644846ac572883c51f23de
I20260809 01:45:55.798761 10591 rpcz_store.cc:276] 0809 01:45:54.176180 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:54.748207 (+572027us) service_pool.cc:224] Handling call
0809 01:45:55.333568 (+585361us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8695846 00007fa2c6252634 00007fa2c62525c2 00007fa2c62524f1 00007fa2c625246c 00007fa2c6251722 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:55.334260 (+ 692us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:55.356708 (+ 22448us) catalog_manager.cc:3751] Apply alter schema
0809 01:45:55.357238 (+ 530us) catalog_manager.cc:3783] Acquired catalog manager lock
0809 01:45:55.357435 (+ 197us) catalog_manager.cc:3941] Updating metadata on disk
0809 01:45:55.404933 (+ 47498us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b5400030300. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c6be3b41 00007fa2c6cb8569 00007fa2c6cae18d 00007fa2c6c6f7eb 00007fa2c6c6f5c4 00007fa2c86ccb51 00007fa2c86cd3af 00007fa2c85806a4 00007fa2c857d7ca 00007fa2c85758bb 00007fa2c8695882
0809 01:45:55.796888 (+391955us) catalog_manager.cc:3972] Committing alterations to in-memory state
0809 01:45:55.798409 (+ 1521us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:55.589991 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:55.590443 (+ 452us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:55.590471 (+ 28us) write_op.cc:435] Acquired schema lock
0809 01:45:55.590487 (+ 16us) tablet.cc:662] Decoding operations
0809 01:45:55.590780 (+ 293us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:55.590818 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:55.590849 (+ 31us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:55.591003 (+ 154us) tablet.cc:701] Row locks acquired
0809 01:45:55.591019 (+ 16us) write_op.cc:260] PREPARE: finished
0809 01:45:55.591144 (+ 125us) write_op.cc:270] Start()
0809 01:45:55.591226 (+ 82us) write_op.cc:276] Timestamp: P: 1786239955590929 usec, L: 0
0809 01:45:55.591243 (+ 17us) op_driver.cc:348] REPLICATION: starting
0809 01:45:55.591751 (+ 508us) log.cc:844] Serialized 405 byte log entry
0809 01:45:55.775898 (+184147us) op_driver.cc:464] REPLICATION: finished
0809 01:45:55.795035 (+ 19137us) write_op.cc:301] APPLY: starting
0809 01:45:55.795107 (+ 72us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:55.795209 (+ 102us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:55.795334 (+ 125us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:55.795568 (+ 234us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:55.795700 (+ 132us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:55.795740 (+ 40us) write_op.cc:312] APPLY: finished
0809 01:45:55.795920 (+ 180us) log.cc:844] Serialized 23 byte log entry
0809 01:45:55.796088 (+ 168us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:55.796164 (+ 76us) write_op.cc:454] Released schema lock
0809 01:45:55.796193 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:55.796452 (+ 259us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:55.796478 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":1362497664,"child_traces":[["op",{"apply.queue_time_us":18916,"apply.run_cpu_time_us":1374,"apply.run_wall_time_us":1375,"num_ops":1,"prepare.queue_time_us":184862,"prepare.run_cpu_time_us":2028,"prepare.run_wall_time_us":32230,"replication_time_us":184415}]]}
I20260809 01:45:55.802934 10867 tablet_service.cc:1511] Processing CreateTablet for tablet c151a35902014f10902fb545d1e646e4 (DEFAULT_TABLE table=default.Table_57b8282c32f843b1882c94f5479bd35a [id=34d18887856a44c0a716874a85e3c49d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.803781 10598 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 412) took 1103 ms. Trace:
I20260809 01:45:55.803956 10867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c151a35902014f10902fb545d1e646e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.804044 10598 rpcz_store.cc:276] 0809 01:45:54.700848 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:54.871154 (+170306us) service_pool.cc:224] Handling call
0809 01:45:55.344859 (+473705us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c86954d6 00007fa2c624db34 00007fa2c624dac2 00007fa2c624d9f1 00007fa2c624d96c 00007fa2c624cc22 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:55.345771 (+ 912us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:55.346760 (+ 989us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:55.346888 (+ 128us) catalog_manager.cc:2850] Locking tablets
0809 01:45:55.347554 (+ 666us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:55.780813 (+433259us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:55.781553 (+ 740us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:55.801633 (+ 20080us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:55.803668 (+ 2035us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:55.543442 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:55.544365 (+ 923us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:55.544419 (+ 54us) write_op.cc:435] Acquired schema lock
0809 01:45:55.544453 (+ 34us) tablet.cc:662] Decoding operations
0809 01:45:55.545065 (+ 612us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:55.545158 (+ 93us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:55.545207 (+ 49us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:55.562022 (+ 16815us) tablet.cc:701] Row locks acquired
0809 01:45:55.562063 (+ 41us) write_op.cc:260] PREPARE: finished
0809 01:45:55.562230 (+ 167us) write_op.cc:270] Start()
0809 01:45:55.562368 (+ 138us) write_op.cc:276] Timestamp: P: 1786239955562017 usec, L: 0
0809 01:45:55.562409 (+ 41us) op_driver.cc:348] REPLICATION: starting
0809 01:45:55.563152 (+ 743us) log.cc:844] Serialized 790 byte log entry
0809 01:45:55.774699 (+211547us) op_driver.cc:464] REPLICATION: finished
0809 01:45:55.776726 (+ 2027us) write_op.cc:301] APPLY: starting
0809 01:45:55.776884 (+ 158us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:55.777099 (+ 215us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:55.777330 (+ 231us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:55.777859 (+ 529us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:55.778126 (+ 267us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:55.778202 (+ 76us) write_op.cc:312] APPLY: finished
0809 01:45:55.778640 (+ 438us) log.cc:844] Serialized 29 byte log entry
0809 01:45:55.780136 (+ 1496us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:55.780269 (+ 133us) write_op.cc:454] Released schema lock
0809 01:45:55.780318 (+ 49us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:55.780677 (+ 359us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:55.780733 (+ 56us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":1059460352,"child_traces":[["op",{"apply.queue_time_us":1656,"apply.run_cpu_time_us":2637,"apply.run_wall_time_us":3933,"num_ops":2,"prepare.queue_time_us":164139,"prepare.run_cpu_time_us":3443,"prepare.run_wall_time_us":21086,"replication_time_us":212049}]]}
I20260809 01:45:55.810563 11119 tablet_service.cc:1558] Processing DeleteTablet for tablet 75e4315b3c644846ac572883c51f23de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:55 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:55.810539 10858 tablet_service.cc:1511] Processing CreateTablet for tablet 2126670a2f29408a9ba4b949f966bf76 (DEFAULT_TABLE table=default.Table_7399370c91a7495faa66e98683c688d0 [id=43255c746b5c4fcc91454fcad2c801a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.811532 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2126670a2f29408a9ba4b949f966bf76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.815907 11149 raft_consensus.cc:1275] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 2 FOLLOWER]: Refusing update from remote peer 4f60c9ac9414403d9ea5a97b103c2316: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260809 01:45:55.816284 11149 pending_rounds.cc:85] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7: Aborting all ops after (but not including) 0
I20260809 01:45:55.816521 11149 pending_rounds.cc:107] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260809 01:45:55.816870 11149 raft_consensus.cc:2889] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 01:45:55.817184 11149 pending_rounds.cc:107] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7: Aborting uncommitted ALTER_SCHEMA_OP operation due to leader change: 1.2
W20260809 01:45:55.819252 11149 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56566: Aborted: Op aborted by new leader
I20260809 01:45:55.819803 11149 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:56566 (request call id 295) took 1906 ms. Trace:
I20260809 01:45:55.820151 11149 rpcz_store.cc:276] 0809 01:45:53.913373 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:53.916850 (+ 3477us) service_pool.cc:224] Handling call
0809 01:45:55.819721 (+1902871us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:54.229854 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0809 01:45:54.231338 (+ 1484us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 01:45:54.231376 (+ 38us) alter_schema_op.cc:66] Acquired schema lock
0809 01:45:54.231434 (+ 58us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0809 01:45:54.231547 (+ 113us) alter_schema_op.cc:128] START. Timestamp: 1786239954231311
0809 01:45:54.231567 (+ 20us) op_driver.cc:348] REPLICATION: starting
0809 01:45:54.241980 (+ 10413us) spinlock_profiling.cc:243] Waited 7.51 ms on lock 0x7b5400202a18. stack: 00007fe4bc3e2cf3 00007fe4bc3e2bf0 00007fe4bbf90574 0000000000516de4 0000000000516d79 00007fe4c431f964 00007fe4c3535d19 00007fe4c34f1a01 00007fe4c34f16da 00007fe4c3575d45 00007fe4c35761e9 00007fe4c3a05f61 00007fe4c3a05738 00007fe4c3a0a3d1 00007fe4c3a0a389 00007fe4c3a0a319
0809 01:45:54.242295 (+ 315us) log.cc:844] Serialized 173 byte log entry
0809 01:45:55.819034 (+1576739us) op_driver.cc:464] REPLICATION: finished
0809 01:45:55.819120 (+ 86us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0809 01:45:55.819164 (+ 44us) alter_schema_op.cc:171] AlterSchemaCommitCallback: op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":302676,"prepare.run_cpu_time_us":8236,"prepare.run_wall_time_us":60702,"replication_time_us":1585900,"spinlock_wait_cycles":17278848,"thread_start_us":37531,"threads_started":1,"wal-append.queue_time_us":36043,"wal-append.run_cpu_time_us":1572,"wal-append.run_wall_time_us":1001632}]]}
I20260809 01:45:55.825766 12920 consensus_queue.cc:1048] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 01:45:55.828245 10576 catalog_manager.cc:5129] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): alter failed for tablet cfb9afd3be194830ba99f5e20fdaca0a (table default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]): Aborted: Op aborted by new leader
I20260809 01:45:55.830303 10862 tablet_service.cc:1511] Processing CreateTablet for tablet b38a0a3033cc4328a0200e11c8eb5c31 (DEFAULT_TABLE table=default.Table_946a0fc83ebc47b99a4d35abd513a966 [id=cccf27c2588f40e79723a91fa276642c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.831406 10862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b38a0a3033cc4328a0200e11c8eb5c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.832530 11551 tablet_bootstrap.cc:492] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:55.832600 11554 ts_tablet_manager.cc:1434] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.467s user 0.002s sys 0.011s
I20260809 01:45:55.832950 11551 ts_tablet_manager.cc:1403] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.201s user 0.009s sys 0.007s
I20260809 01:45:55.834115 10845 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:45966 (request call id 49) took 1134 ms. Trace:
I20260809 01:45:55.834508 10845 rpcz_store.cc:276] 0809 01:45:54.700031 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:54.700137 (+ 106us) service_pool.cc:224] Handling call
0809 01:45:55.834057 (+1133920us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:55.842049 12983 tablet.cc:1724] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:55.842896 10847 tablet_service.cc:1511] Processing CreateTablet for tablet d8ca63cedd3b4626949a0ec07bc26290 (DEFAULT_TABLE table=default.Table_199eacafb2d74a8190d24018b14db258 [id=e224e231c3e64f1ca6887a3593acf948]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.844164 10847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8ca63cedd3b4626949a0ec07bc26290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.844195 11554 tablet_bootstrap.cc:492] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
W20260809 01:45:55.849637 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 144437us
I20260809 01:45:55.854060 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet 75e4315b3c644846ac572883c51f23de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:55 UTC) from {username='slave'} at 127.0.0.1:45982
W20260809 01:45:55.855223 10804 leader_election.cc:343] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:55.857990 10905 raft_consensus.cc:1275] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 2 FOLLOWER]: Refusing update from remote peer 4f60c9ac9414403d9ea5a97b103c2316: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260809 01:45:55.858403 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 6b94950efa2a49588fc2a69082e33acb
I20260809 01:45:55.859889 10597 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 410) took 1334 ms. Trace:
I20260809 01:45:55.862951 10592 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_c54e4ecef7c145cc970af6169c5a9326 [id=40cb4403a8a94bd3938983441e9254d4] alter complete (version 1)
I20260809 01:45:55.858470 11551 raft_consensus.cc:359] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.863531 11551 raft_consensus.cc:385] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:55.863785 11551 raft_consensus.cc:740] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
W20260809 01:45:55.864223 10592 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 55) took 947 ms (client timeout 1000 ms). Trace:
I20260809 01:45:55.864396 11551 consensus_queue.cc:260] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.866302 11551 ts_tablet_manager.cc:1434] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.033s user 0.002s sys 0.005s
W20260809 01:45:55.864512 10592 rpcz_store.cc:269] 0809 01:45:54.916897 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:55.245620 (+328723us) service_pool.cc:224] Handling call
0809 01:45:55.333957 (+ 88337us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:55.864133 (+530176us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:55.655221 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:55.655658 (+ 437us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:55.655685 (+ 27us) write_op.cc:435] Acquired schema lock
0809 01:45:55.655701 (+ 16us) tablet.cc:662] Decoding operations
0809 01:45:55.655983 (+ 282us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:55.656019 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:55.656046 (+ 27us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:55.656210 (+ 164us) tablet.cc:701] Row locks acquired
0809 01:45:55.656226 (+ 16us) write_op.cc:260] PREPARE: finished
0809 01:45:55.656340 (+ 114us) write_op.cc:270] Start()
0809 01:45:55.656419 (+ 79us) write_op.cc:276] Timestamp: P: 1786239955656121 usec, L: 0
0809 01:45:55.656433 (+ 14us) op_driver.cc:348] REPLICATION: starting
0809 01:45:55.656927 (+ 494us) log.cc:844] Serialized 356 byte log entry
0809 01:45:55.855060 (+198133us) op_driver.cc:464] REPLICATION: finished
0809 01:45:55.857062 (+ 2002us) write_op.cc:301] APPLY: starting
0809 01:45:55.857142 (+ 80us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:55.857230 (+ 88us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:55.857310 (+ 80us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:55.857461 (+ 151us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:55.861940 (+ 4479us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:55.861979 (+ 39us) write_op.cc:312] APPLY: finished
0809 01:45:55.862145 (+ 166us) log.cc:844] Serialized 23 byte log entry
0809 01:45:55.862334 (+ 189us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:55.862400 (+ 66us) write_op.cc:454] Released schema lock
0809 01:45:55.862432 (+ 32us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":2,"spinlock_wait_cycles":5122304,"child_traces":[["op",{"apply.queue_time_us":1871,"apply.run_cpu_time_us":1273,"apply.run_wall_time_us":8600,"num_ops":1,"prepare.queue_time_us":215844,"prepare.run_cpu_time_us":1924,"prepare.run_wall_time_us":2794,"replication_time_us":198544}]]}
I20260809 01:45:55.869107 11551 tablet_bootstrap.cc:492] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:55.857491 12904 ts_tablet_manager.cc:1929] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:55.860147 10597 rpcz_store.cc:276] 0809 01:45:54.525235 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:54.821463 (+296228us) service_pool.cc:224] Handling call
0809 01:45:55.343247 (+521784us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c86954d6 00007fa2c624db34 00007fa2c624dac2 00007fa2c624d9f1 00007fa2c624d96c 00007fa2c624cc22 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:55.344221 (+ 974us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:55.386833 (+ 42612us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:55.386953 (+ 120us) catalog_manager.cc:2850] Locking tablets
0809 01:45:55.414151 (+ 27198us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:55.857730 (+443579us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:55.858096 (+ 366us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:55.858186 (+ 90us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:55.859832 (+ 1646us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:55.622408 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:55.622893 (+ 485us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:55.622934 (+ 41us) write_op.cc:435] Acquired schema lock
0809 01:45:55.622947 (+ 13us) tablet.cc:662] Decoding operations
0809 01:45:55.623313 (+ 366us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:55.623354 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:55.623383 (+ 29us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:55.623721 (+ 338us) tablet.cc:701] Row locks acquired
0809 01:45:55.623740 (+ 19us) write_op.cc:260] PREPARE: finished
0809 01:45:55.623861 (+ 121us) write_op.cc:270] Start()
0809 01:45:55.623957 (+ 96us) write_op.cc:276] Timestamp: P: 1786239955623641 usec, L: 0
0809 01:45:55.623978 (+ 21us) op_driver.cc:348] REPLICATION: starting
0809 01:45:55.654112 (+ 30134us) log.cc:844] Serialized 725 byte log entry
0809 01:45:55.654961 (+ 849us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678d951 00007fa2c6820208 00007fa2c6caff61 00007fa2c6caf738 00007fa2c6cb43d1 00007fa2c6cb4389 00007fa2c6cb4319 00007fa2c6cb42e1 00007fa2c6cb350d 00000000005bd8c4 00000000005bd6f9
0809 01:45:55.854785 (+199824us) op_driver.cc:464] REPLICATION: finished
0809 01:45:55.855299 (+ 514us) write_op.cc:301] APPLY: starting
0809 01:45:55.855365 (+ 66us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:55.855502 (+ 137us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:55.855526 (+ 24us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:55.855862 (+ 336us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:55.856012 (+ 150us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:55.856054 (+ 42us) write_op.cc:312] APPLY: finished
0809 01:45:55.856261 (+ 207us) log.cc:844] Serialized 29 byte log entry
0809 01:45:55.856394 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:55.856472 (+ 78us) write_op.cc:454] Released schema lock
0809 01:45:55.856517 (+ 45us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:55.856742 (+ 225us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:55.856764 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":1122411392,"child_traces":[["op",{"apply.queue_time_us":330,"apply.run_cpu_time_us":1413,"apply.run_wall_time_us":1414,"num_ops":2,"prepare.queue_time_us":206231,"prepare.run_cpu_time_us":3245,"prepare.run_wall_time_us":32681,"replication_time_us":230671,"spinlock_wait_cycles":67952512}]]}
I20260809 01:45:55.873832 12904 log.cc:1199] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/c528b45de0d14c19af5d6836b7a14f1c
I20260809 01:45:55.877244 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e03033672e947eab8cc143263281133" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:55.878032 11551 tablet_bootstrap.cc:654] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:55.879058 10805 leader_election.cc:343] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:55.879441 10805 leader_election.cc:304] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:55.880436 12889 raft_consensus.cc:2749] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:55.880564 12904 ts_tablet_manager.cc:1950] T c528b45de0d14c19af5d6836b7a14f1c P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:55.883414 12904 tablet_replica.cc:333] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:55.885149 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet c528b45de0d14c19af5d6836b7a14f1c (table default.Table_53624396796f4684acbc3787e6e05f6e [id=18e87d9e1fa3442aaa489e3abcd4640e]) successfully deleted
W20260809 01:45:55.875576 11289 leader_election.cc:343] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 197b5043f6d348f89853c073b256daa4
I20260809 01:45:55.887710 11105 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b94950efa2a49588fc2a69082e33acb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:55 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:55.887913 11289 leader_election.cc:304] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:55.889204 12986 raft_consensus.cc:2749] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:55.890084 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b94950efa2a49588fc2a69082e33acb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:55 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:55.898422 11553 consensus_queue.cc:260] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.906226 10858 tablet_service.cc:1511] Processing CreateTablet for tablet 01e9f79113fb48c58e7fdbfab4fb49b8 (DEFAULT_TABLE table=default.Table_d5bd0ddd910f400b88147ea461999b39 [id=db57dff908bf4fc38861e80de59e00c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.907372 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01e9f79113fb48c58e7fdbfab4fb49b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.912438 12986 consensus_queue.cc:1048] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:55.912456 12904 raft_consensus.cc:2243] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:55.916769 12904 raft_consensus.cc:2272] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:55.914152 12889 raft_consensus.cc:493] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:55.919297 12904 ts_tablet_manager.cc:1916] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:55.919129 12889 raft_consensus.cc:515] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.920964 12889 leader_election.cc:290] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:55.922626 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fffa48ccf58476da79dccd9c008b67c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:55.922878 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fffa48ccf58476da79dccd9c008b67c" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:55.923972 10805 leader_election.cc:343] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:55.924360 10804 leader_election.cc:343] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:55.924835 10804 leader_election.cc:304] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:55.925351 11551 tablet_bootstrap.cc:492] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:55.925702 12889 raft_consensus.cc:2749] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:55.930642 11553 ts_tablet_manager.cc:1434] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.282s user 0.007s sys 0.000s
I20260809 01:45:55.931983 11553 tablet_bootstrap.cc:492] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:55.950112 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 60d356fcc48441ca80f2df29b533404f (table default.Table_29a4bd6bd10a473dbd9dbda008b9eb16 [id=67ee87fabc7749279341528fbcce16d4]) successfully deleted
I20260809 01:45:55.952559 11346 tablet_service.cc:1511] Processing CreateTablet for tablet b93faf596f3c4dd286e600013c8db001 (DEFAULT_TABLE table=default.Table_79eb7d6b90da4bccbad721f2e840dd31 [id=87531d380b21454493fdffbcdc4ea825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.953729 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b93faf596f3c4dd286e600013c8db001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.927218 11551 ts_tablet_manager.cc:1403] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.058s user 0.008s sys 0.003s
I20260809 01:45:55.976389 12889 raft_consensus.cc:493] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:55.976847 12889 raft_consensus.cc:515] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.978658 12889 leader_election.cc:290] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:55.961375 11551 raft_consensus.cc:359] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.979727 11551 raft_consensus.cc:385] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:55.980036 11551 raft_consensus.cc:740] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:55.980038 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a238ad3d46054daa9435e9076ba8311d" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:55.981177 10805 leader_election.cc:343] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:55.982780 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a238ad3d46054daa9435e9076ba8311d" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:55.985177 10804 leader_election.cc:343] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:55.985608 10804 leader_election.cc:304] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:55.986263 12889 raft_consensus.cc:2749] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:55.980638 11551 consensus_queue.cc:260] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:55.988629 11551 ts_tablet_manager.cc:1434] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.029s user 0.003s sys 0.001s
I20260809 01:45:55.988406 11360 tablet_service.cc:1511] Processing CreateTablet for tablet b38a0a3033cc4328a0200e11c8eb5c31 (DEFAULT_TABLE table=default.Table_946a0fc83ebc47b99a4d35abd513a966 [id=cccf27c2588f40e79723a91fa276642c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:55.989568 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b38a0a3033cc4328a0200e11c8eb5c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.991724 12778 consensus_queue.cc:1048] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:55.995317 11551 tablet_bootstrap.cc:492] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:56.007066 11366 tablet_service.cc:1511] Processing CreateTablet for tablet c151a35902014f10902fb545d1e646e4 (DEFAULT_TABLE table=default.Table_57b8282c32f843b1882c94f5479bd35a [id=34d18887856a44c0a716874a85e3c49d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:56.008185 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c151a35902014f10902fb545d1e646e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:56.007874 11336 tablet_service.cc:1511] Processing CreateTablet for tablet 01e9f79113fb48c58e7fdbfab4fb49b8 (DEFAULT_TABLE table=default.Table_d5bd0ddd910f400b88147ea461999b39 [id=db57dff908bf4fc38861e80de59e00c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:56.009112 11336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01e9f79113fb48c58e7fdbfab4fb49b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:56.010234 11341 tablet_service.cc:1511] Processing CreateTablet for tablet 2126670a2f29408a9ba4b949f966bf76 (DEFAULT_TABLE table=default.Table_7399370c91a7495faa66e98683c688d0 [id=43255c746b5c4fcc91454fcad2c801a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:56.011317 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2126670a2f29408a9ba4b949f966bf76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:55.974243 11949 ts_tablet_manager.cc:1929] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:56.029573 11349 tablet_service.cc:1558] Processing DeleteTablet for tablet 36a542c773b7438ead16c5b3af224cf9 with delete_type TABLET_DATA_DELETED (replaced by tablet a38bd344108649ed8e1687f82154000b) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:56.030128 11949 log.cc:1199] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/1c73bed7a9774cbc8ac774e6514a981d
I20260809 01:45:56.030839 11949 ts_tablet_manager.cc:1950] T 1c73bed7a9774cbc8ac774e6514a981d P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:56.032667 11349 tablet_service.cc:1558] Processing DeleteTablet for tablet b84f1943c1a34e999fc7c12a5847cdb6 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ed45a2d5e9b4ad8be6f94e4a895003f) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:56.034137 11349 tablet_service.cc:1511] Processing CreateTablet for tablet d8ca63cedd3b4626949a0ec07bc26290 (DEFAULT_TABLE table=default.Table_199eacafb2d74a8190d24018b14db258 [id=e224e231c3e64f1ca6887a3593acf948]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:56.035552 11349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8ca63cedd3b4626949a0ec07bc26290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:56.046555 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 277) took 5130 ms. Trace:
I20260809 01:45:56.046860 11949 rpcz_store.cc:276] 0809 01:45:50.916492 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:50.916617 (+ 125us) service_pool.cc:224] Handling call
0809 01:45:55.584866 (+4668249us) ts_tablet_manager.cc:1151] Deleting tablet 1c73bed7a9774cbc8ac774e6514a981d
0809 01:45:55.584923 (+ 57us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:55.816183 (+231260us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:55.974128 (+157945us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:56.046483 (+ 72355us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4647124}
I20260809 01:45:56.048115 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 1c73bed7a9774cbc8ac774e6514a981d (table default.Table_4657bf5b97b244a19b9e1f6c242a6552 [id=ac5e8f07619c446aa7f6249afd741e14]) successfully deleted
I20260809 01:45:56.052598 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 283) took 4562 ms. Trace:
I20260809 01:45:56.053933 12874 tablet_replica.cc:333] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:56.054858 12874 raft_consensus.cc:2243] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:56.055384 12874 raft_consensus.cc:2272] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:56.057933 12874 ts_tablet_manager.cc:1916] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:56.060963 11949 rpcz_store.cc:276] 0809 01:45:51.490139 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.490412 (+ 273us) service_pool.cc:224] Handling call
0809 01:45:56.052283 (+4561871us) ts_tablet_manager.cc:1151] Deleting tablet 991e8580b8624f34881d56c6cc60600f
0809 01:45:56.052322 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:56.052549 (+ 227us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4560563}
W20260809 01:45:56.062942 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 991e8580b8624f34881d56c6cc60600f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 991e8580b8624f34881d56c6cc60600f already in progress: creating tablet
I20260809 01:45:56.063547 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 286) took 4437 ms. Trace:
I20260809 01:45:56.063853 11949 rpcz_store.cc:276] 0809 01:45:51.626035 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:51.626353 (+ 318us) service_pool.cc:224] Handling call
0809 01:45:56.063219 (+4436866us) ts_tablet_manager.cc:1151] Deleting tablet 7634fe763a4e4fec91c5938e45a5d1fc
0809 01:45:56.063260 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:56.063497 (+ 237us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4435533}
W20260809 01:45:56.065919 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 7634fe763a4e4fec91c5938e45a5d1fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7634fe763a4e4fec91c5938e45a5d1fc already in progress: creating tablet
I20260809 01:45:56.067301 11949 tablet_replica.cc:333] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:56.068078 11949 raft_consensus.cc:2243] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:56.068581 11949 raft_consensus.cc:2272] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:56.070492 11949 ts_tablet_manager.cc:1916] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:56.070955 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_2782952361d24cf9aa19e0667a8a67ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:56.072501 11553 tablet_bootstrap.cc:654] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:56.073076 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2782952361d24cf9aa19e0667a8a67ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:56.074523 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_a5ecac2f48c1491fb074981ce05ef950"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:56.076262 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a5ecac2f48c1491fb074981ce05ef950 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:56.083801 10596 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_218ebdf2aa774da6b7b15e22e0d703df" } new_table_name: "default.Table_b2aa39a9e9b941bbb2e8b36250125c06" modify_external_catalogs: true
I20260809 01:45:56.086355 10586 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_41c3d9aefb8b49a1afe6c6481fd8fd2f" } modify_external_catalogs: true
I20260809 01:45:56.086835 10586 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_41c3d9aefb8b49a1afe6c6481fd8fd2f" } modify_external_catalogs: true
I20260809 01:45:56.092113 11554 tablet_bootstrap.cc:700] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316: Time spent opening tablet: real 0.149s user 0.000s sys 0.000s
I20260809 01:45:56.092854 11554 tablet_bootstrap.cc:654] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:56.092542 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_eb837ff4a9784eed8fca586c6c6b53ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:56.098222 10588 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_55fdbc6110a64bed9f6caafd0b681303" } new_table_name: "default.Table_d3d0147188b44c36a28f4aba4e439350" modify_external_catalogs: true
W20260809 01:45:56.101686 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb837ff4a9784eed8fca586c6c6b53ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:56.104125 11949 ts_tablet_manager.cc:1929] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:56.104465 11949 log.cc:1199] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/b9493892c1ff48e7a0558ff0fcb7a0f3
I20260809 01:45:56.105103 11949 ts_tablet_manager.cc:1950] T b9493892c1ff48e7a0558ff0fcb7a0f3 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:56.106998 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 296) took 3116 ms. Trace:
I20260809 01:45:56.107283 11949 rpcz_store.cc:276] 0809 01:45:52.991159 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:52.991915 (+ 756us) service_pool.cc:224] Handling call
0809 01:45:56.067156 (+3075241us) ts_tablet_manager.cc:1151] Deleting tablet b9493892c1ff48e7a0558ff0fcb7a0f3
0809 01:45:56.067197 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:56.096949 (+ 29752us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:56.104017 (+ 7068us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:56.106944 (+ 2927us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3074643}
I20260809 01:45:56.108439 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet b9493892c1ff48e7a0558ff0fcb7a0f3 (table default.Table_a1e1833d52934d0b90c8e6f5046d03df [id=59cbeb450bbc4fb99011118dbfe5809a]) successfully deleted
I20260809 01:45:56.108903 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 297) took 3117 ms. Trace:
I20260809 01:45:56.109194 11949 rpcz_store.cc:276] 0809 01:45:52.991663 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:52.992610 (+ 947us) service_pool.cc:224] Handling call
0809 01:45:56.108529 (+3115919us) ts_tablet_manager.cc:1151] Deleting tablet 012b043d7eef472eb30bdd8e3eec271f
0809 01:45:56.108566 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:56.108854 (+ 288us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3115430}
I20260809 01:45:56.110666 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 298) took 3008 ms. Trace:
I20260809 01:45:56.110953 11949 rpcz_store.cc:276] 0809 01:45:53.102463 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:53.103079 (+ 616us) service_pool.cc:224] Handling call
0809 01:45:56.110311 (+3007232us) ts_tablet_manager.cc:1151] Deleting tablet b095338eb59e4161aaaf3afb762cd296
0809 01:45:56.110349 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:56.110597 (+ 248us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3006723}
W20260809 01:45:56.112088 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet b095338eb59e4161aaaf3afb762cd296 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b095338eb59e4161aaaf3afb762cd296 already in progress: creating tablet
I20260809 01:45:56.112393 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 320) took 1363 ms. Trace:
I20260809 01:45:56.112661 11949 rpcz_store.cc:276] 0809 01:45:54.749156 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:54.750609 (+ 1453us) service_pool.cc:224] Handling call
0809 01:45:56.112071 (+1361462us) ts_tablet_manager.cc:1151] Deleting tablet 2790e2266ade4c698741a940ed49757e
0809 01:45:56.112108 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:56.112322 (+ 214us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1277151}
W20260809 01:45:56.113754 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 2790e2266ade4c698741a940ed49757e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2790e2266ade4c698741a940ed49757e already in progress: creating tablet
I20260809 01:45:56.114092 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 322) took 1349 ms. Trace:
I20260809 01:45:56.114395 11949 rpcz_store.cc:276] 0809 01:45:54.765074 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:54.765406 (+ 332us) service_pool.cc:224] Handling call
0809 01:45:56.113791 (+1348385us) ts_tablet_manager.cc:1151] Deleting tablet 4aa1429f21024739b06bec88c3cb24d9
0809 01:45:56.113831 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:56.114046 (+ 215us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1277139}
W20260809 01:45:56.115435 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 4aa1429f21024739b06bec88c3cb24d9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4aa1429f21024739b06bec88c3cb24d9 already in progress: creating tablet
I20260809 01:45:56.116945 11949 tablet_replica.cc:333] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:56.117707 11949 raft_consensus.cc:2243] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:56.118124 11949 raft_consensus.cc:2272] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:56.119889 11949 ts_tablet_manager.cc:1916] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:56.131263 11553 tablet_bootstrap.cc:492] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:56.131698 11553 ts_tablet_manager.cc:1403] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.200s user 0.008s sys 0.000s
I20260809 01:45:56.117352 10592 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet b6159d660d544b3e907a08372027c0e2 (table default.Table_99bad24613e94b6f840f637bea4978fc [id=ddfe680283854b9484dc3e9d6e626ab9]). Expected version 1 got 0
I20260809 01:45:56.135377 10592 catalog_manager.cc:5719] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 reported cstate change: term changed from 1 to 2, leader changed from 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66) to 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67). New cstate: current_term: 2 leader_uuid: "4f60c9ac9414403d9ea5a97b103c2316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:56.136651 10592 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet cfb9afd3be194830ba99f5e20fdaca0a (table default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]). Expected version 1 got 0
I20260809 01:45:56.142568 11553 raft_consensus.cc:359] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.143218 11553 raft_consensus.cc:385] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:56.143491 11553 raft_consensus.cc:740] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:56.144623 11949 ts_tablet_manager.cc:1929] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:56.145031 11949 log.cc:1199] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/9fffa48ccf58476da79dccd9c008b67c
I20260809 01:45:56.145769 11553 consensus_queue.cc:260] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.146827 11949 ts_tablet_manager.cc:1950] T 9fffa48ccf58476da79dccd9c008b67c P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:56.151623 11553 ts_tablet_manager.cc:1434] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.020s user 0.006s sys 0.002s
I20260809 01:45:56.166970 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 328) took 1244 ms. Trace:
I20260809 01:45:56.167280 11949 rpcz_store.cc:276] 0809 01:45:54.922754 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:54.922955 (+ 201us) service_pool.cc:224] Handling call
0809 01:45:56.116809 (+1193854us) ts_tablet_manager.cc:1151] Deleting tablet 9fffa48ccf58476da79dccd9c008b67c
0809 01:45:56.116847 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:56.127772 (+ 10925us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:56.144519 (+ 16747us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:56.166899 (+ 22380us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1192886}
I20260809 01:45:56.168395 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 9fffa48ccf58476da79dccd9c008b67c (table default.Table_3227dde355ef4460a73ac13726bf7249 [id=3e51bbb9edf142ecabc9c92777fd80b4]) successfully deleted
I20260809 01:45:56.168833 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 327) took 1271 ms. Trace:
I20260809 01:45:56.169117 11949 rpcz_store.cc:276] 0809 01:45:54.922149 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:54.946970 (+ 24821us) service_pool.cc:224] Handling call
0809 01:45:56.168187 (+1221217us) ts_tablet_manager.cc:1151] Deleting tablet a85e82d85424445d94b30402d41c53dc
0809 01:45:56.168224 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:56.168774 (+ 550us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1219956}
I20260809 01:45:56.170104 11949 tablet_replica.cc:333] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:56.170677 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet efe4c7b15604442eb7be83fe19bb54f4
I20260809 01:45:56.171588 11553 tablet_bootstrap.cc:492] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:56.172539 10589 catalog_manager.cc:6807] ReplaceTablet: tablet efe4c7b15604442eb7be83fe19bb54f4 deleted and replaced by tablet 39b00ef870e94bee8a9bd247e69400a6
I20260809 01:45:56.175176 11360 tablet_service.cc:1558] Processing DeleteTablet for tablet 75e4315b3c644846ac572883c51f23de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:55 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:56.175835 11336 tablet_service.cc:1558] Processing DeleteTablet for tablet efe4c7b15604442eb7be83fe19bb54f4 with delete_type TABLET_DATA_DELETED (replaced by tablet 39b00ef870e94bee8a9bd247e69400a6) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:56.176513 11949 raft_consensus.cc:2243] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:56.177012 11949 raft_consensus.cc:2272] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:56.178035 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet a85e82d85424445d94b30402d41c53dc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a85e82d85424445d94b30402d41c53dc already in progress: creating tablet
I20260809 01:45:56.179872 10858 tablet_service.cc:1558] Processing DeleteTablet for tablet efe4c7b15604442eb7be83fe19bb54f4 with delete_type TABLET_DATA_DELETED (replaced by tablet 39b00ef870e94bee8a9bd247e69400a6) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:56.189452 11553 tablet_bootstrap.cc:654] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:56.189937 10589 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_a21564a1b68c4bc9b78278dfa70ad1a9" } modify_external_catalogs: true
I20260809 01:45:56.190477 10589 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_a21564a1b68c4bc9b78278dfa70ad1a9" } modify_external_catalogs: true
I20260809 01:45:56.192039 11949 ts_tablet_manager.cc:1916] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:56.194985 12889 raft_consensus.cc:493] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:56.195422 12889 raft_consensus.cc:515] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.197245 12889 leader_election.cc:290] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:56.198843 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b31998b65e4264b16c0d73dbcfc73b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:56.200273 10805 leader_election.cc:343] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:56.201200 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b94950efa2a49588fc2a69082e33acb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:55 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:56.211440 12874 ts_tablet_manager.cc:1929] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:56.211975 12874 log.cc:1199] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/36a542c773b7438ead16c5b3af224cf9
I20260809 01:45:56.212885 12874 ts_tablet_manager.cc:1950] T 36a542c773b7438ead16c5b3af224cf9 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:56.216037 12874 tablet_replica.cc:333] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:56.217037 12874 raft_consensus.cc:2243] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:56.218966 11105 tablet_service.cc:1558] Processing DeleteTablet for tablet efe4c7b15604442eb7be83fe19bb54f4 with delete_type TABLET_DATA_DELETED (replaced by tablet 39b00ef870e94bee8a9bd247e69400a6) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:56.220849 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 36a542c773b7438ead16c5b3af224cf9 (table default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]) successfully deleted
I20260809 01:45:56.229653 11554 tablet_bootstrap.cc:492] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:56.230052 11554 ts_tablet_manager.cc:1403] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.386s user 0.010s sys 0.008s
I20260809 01:45:56.232335 11554 raft_consensus.cc:359] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.232900 11554 raft_consensus.cc:385] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:56.233129 11554 raft_consensus.cc:740] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:56.233981 11949 ts_tablet_manager.cc:1929] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:56.233958 11554 consensus_queue.cc:260] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.234419 11949 log.cc:1199] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/bd3662c307ba4ff78b8f3165b23acb1a
I20260809 01:45:56.236415 11554 ts_tablet_manager.cc:1434] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
I20260809 01:45:56.236941 12874 raft_consensus.cc:2272] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:56.240041 11554 tablet_bootstrap.cc:492] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:56.162158 12778 raft_consensus.cc:493] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:56.250082 12778 raft_consensus.cc:515] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.250658 11949 ts_tablet_manager.cc:1950] T bd3662c307ba4ff78b8f3165b23acb1a P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:56.252517 12778 leader_election.cc:290] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:56.253495 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 332) took 1014 ms. Trace:
I20260809 01:45:56.253925 11949 rpcz_store.cc:276] 0809 01:45:55.239349 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:55.241358 (+ 2009us) service_pool.cc:224] Handling call
0809 01:45:56.169970 (+928612us) ts_tablet_manager.cc:1151] Deleting tablet bd3662c307ba4ff78b8f3165b23acb1a
0809 01:45:56.170007 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:56.206518 (+ 36511us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:56.233854 (+ 27336us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:56.253422 (+ 19568us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":927960}
I20260809 01:45:56.255191 11949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 331) took 1016 ms. Trace:
I20260809 01:45:56.255486 11949 rpcz_store.cc:276] 0809 01:45:55.238913 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:55.461486 (+222573us) service_pool.cc:224] Handling call
0809 01:45:56.254906 (+793420us) ts_tablet_manager.cc:1151] Deleting tablet 72b4850de7d740d794283ca93152fdc1
0809 01:45:56.254964 (+ 58us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:56.255138 (+ 174us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":738461}
I20260809 01:45:56.258579 12874 ts_tablet_manager.cc:1916] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:56.260190 11554 tablet_bootstrap.cc:654] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:56.261301 11551 tablet_bootstrap.cc:654] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:56.256389 11949 tablet_replica.cc:333] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:56.270330 11949 raft_consensus.cc:2243] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:56.270839 11949 raft_consensus.cc:2272] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:56.283615 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet bd3662c307ba4ff78b8f3165b23acb1a (table default.Table_f97a006f17844462abd4433315667e68 [id=0a517ec159a64c28aee43c00ca3f5359]) successfully deleted
I20260809 01:45:56.284233 11949 ts_tablet_manager.cc:1916] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:56.284364 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 72b4850de7d740d794283ca93152fdc1 because the tablet was not found. No further retry: Not found: Tablet not found: 72b4850de7d740d794283ca93152fdc1
I20260809 01:45:56.286309 13041 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.8.125:42063 every 8 connection(s)
I20260809 01:45:56.287083 11743 rpc_server.cc:307] RPC server started. Bound to: 127.10.8.125:42063
I20260809 01:45:56.292147 11743 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/master-1/data/info.pb
I20260809 01:45:56.299515 10273 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu as pid 11743
I20260809 01:45:56.309916 12874 ts_tablet_manager.cc:1929] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:56.318526 12874 log.cc:1199] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/b84f1943c1a34e999fc7c12a5847cdb6
I20260809 01:45:56.322391 12874 ts_tablet_manager.cc:1950] T b84f1943c1a34e999fc7c12a5847cdb6 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:56.330881 10595 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_99bad24613e94b6f840f637bea4978fc [id=ddfe680283854b9484dc3e9d6e626ab9] alter complete (version 1)
W20260809 01:45:56.336186 10595 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.66:57751 (request call id 63) took 906 ms (client timeout 1000 ms). Trace:
I20260809 01:45:56.337148 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet b84f1943c1a34e999fc7c12a5847cdb6 (table default.Table_e39314f303ee429f897d4f85b8edbf69 [id=21bc1ee96b724341904d6b0c5b10fb7a]) successfully deleted
I20260809 01:45:56.340415 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet 7803a6aff02d42c1833f9b2cd7332b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:56.338991 10597 master_service.cc:960] ReplaceTablet: received request to replace tablet 7706838afc9f4dbebc3213b82c4da126 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:56.348009 12874 tablet_replica.cc:333] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:56.348869 12874 raft_consensus.cc:2243] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:56.349800 12874 raft_consensus.cc:2272] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:56.356451 11949 ts_tablet_manager.cc:1929] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:56.356788 11949 log.cc:1199] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/75e4315b3c644846ac572883c51f23de
I20260809 01:45:56.359050 12874 ts_tablet_manager.cc:1916] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:56.367003 11949 ts_tablet_manager.cc:1950] T 75e4315b3c644846ac572883c51f23de P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:56.369637 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 75e4315b3c644846ac572883c51f23de (table default.Table_ee7c1665ed2344278d1ee031c67438a4 [id=850f362a7f9a4531a3d7d6de29b695ba]) successfully deleted
I20260809 01:45:56.374399 10586 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 4c351957f82d4a58badd2d7d0ae41f81
I20260809 01:45:56.376443 12889 raft_consensus.cc:493] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:56.376856 12889 raft_consensus.cc:515] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:56.378865 12889 leader_election.cc:290] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:56.380470 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325e3d19b83545a197c2e09fc9643b5e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:56.381776 10805 leader_election.cc:343] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:56.385114 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 6b94950efa2a49588fc2a69082e33acb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6b94950efa2a49588fc2a69082e33acb already in progress: creating tablet
W20260809 01:45:56.387153 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet efe4c7b15604442eb7be83fe19bb54f4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet efe4c7b15604442eb7be83fe19bb54f4 already in progress: creating tablet
W20260809 01:45:56.389448 11063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 227561us
I20260809 01:45:56.403570 13048 sys_catalog.cc:263] Verifying existing consensus state
I20260809 01:45:56.404989 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91941616dfdd4df3aaa7c8460876d273" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:56.407171 11062 leader_election.cc:343] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:56.407460 12889 raft_consensus.cc:493] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:56.408051 11062 consensus_peers.cc:597] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:56.407909 12889 raft_consensus.cc:515] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:56.336506 10595 rpcz_store.cc:269] 0809 01:45:55.429696 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:55.509778 (+ 80082us) service_pool.cc:224] Handling call
0809 01:45:55.594998 (+ 85220us) spinlock_profiling.cc:243] Waited 82.7 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:56.336109 (+741111us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:55.821917 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:55.822571 (+ 654us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:55.822617 (+ 46us) write_op.cc:435] Acquired schema lock
0809 01:45:55.822635 (+ 18us) tablet.cc:662] Decoding operations
0809 01:45:55.823095 (+ 460us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:55.823199 (+ 104us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:55.823243 (+ 44us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:55.823617 (+ 374us) tablet.cc:701] Row locks acquired
0809 01:45:55.823640 (+ 23us) write_op.cc:260] PREPARE: finished
0809 01:45:55.872597 (+ 48957us) write_op.cc:270] Start()
0809 01:45:55.872686 (+ 89us) write_op.cc:276] Timestamp: P: 1786239955872353 usec, L: 0
0809 01:45:55.872705 (+ 19us) op_driver.cc:348] REPLICATION: starting
0809 01:45:55.874239 (+ 1534us) log.cc:844] Serialized 901 byte log entry
0809 01:45:55.927809 (+ 53570us) op_driver.cc:464] REPLICATION: finished
0809 01:45:55.930310 (+ 2501us) write_op.cc:301] APPLY: starting
0809 01:45:55.930375 (+ 65us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:55.930504 (+ 129us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:55.930621 (+ 117us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:55.930983 (+ 362us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:55.931169 (+ 186us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:55.931234 (+ 65us) write_op.cc:312] APPLY: finished
0809 01:45:55.931449 (+ 215us) log.cc:844] Serialized 29 byte log entry
0809 01:45:55.931601 (+ 152us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:55.931662 (+ 61us) write_op.cc:454] Released schema lock
0809 01:45:55.931696 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:55.937871 (+ 6175us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:55.937911 (+ 40us) write_op.cc:454] Released schema lock
Related trace 'op':
0809 01:45:55.967789 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:55.968243 (+ 454us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:55.968273 (+ 30us) write_op.cc:435] Acquired schema lock
0809 01:45:55.968289 (+ 16us) tablet.cc:662] Decoding operations
0809 01:45:55.968589 (+ 300us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:55.968630 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:55.968656 (+ 26us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:55.968823 (+ 167us) tablet.cc:701] Row locks acquired
0809 01:45:55.968840 (+ 17us) write_op.cc:260] PREPARE: finished
0809 01:45:55.968962 (+ 122us) write_op.cc:270] Start()
0809 01:45:55.969048 (+ 86us) write_op.cc:276] Timestamp: P: 1786239955968716 usec, L: 0
0809 01:45:55.969066 (+ 18us) op_driver.cc:348] REPLICATION: starting
0809 01:45:56.119195 (+150129us) log.cc:844] Serialized 356 byte log entry
0809 01:45:56.120698 (+ 1503us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678d951 00007fa2c6820208 00007fa2c6caff61 00007fa2c6caf738 00007fa2c6cb43d1 00007fa2c6cb4389 00007fa2c6cb4319 00007fa2c6cb42e1 00007fa2c6cb350d 00000000005bd8c4 00000000005bd6f9
0809 01:45:56.217334 (+ 96636us) op_driver.cc:464] REPLICATION: finished
0809 01:45:56.328535 (+111201us) write_op.cc:301] APPLY: starting
0809 01:45:56.328621 (+ 86us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:56.328734 (+ 113us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:56.328764 (+ 30us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:56.328990 (+ 226us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:56.329161 (+ 171us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:56.329222 (+ 61us) write_op.cc:312] APPLY: finished
0809 01:45:56.329436 (+ 214us) log.cc:844] Serialized 23 byte log entry
0809 01:45:56.329637 (+ 201us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:56.329711 (+ 74us) write_op.cc:454] Released schema lock
0809 01:45:56.329761 (+ 50us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":6,"spinlock_wait_cycles":190291200,"child_traces":[["op",{"apply.queue_time_us":2342,"apply.run_cpu_time_us":1643,"apply.run_wall_time_us":7490,"num_ops":2,"prepare.queue_time_us":7728,"prepare.run_cpu_time_us":2877,"prepare.run_wall_time_us":59636,"replication_time_us":55014}],["op",{"apply.queue_time_us":111008,"apply.run_cpu_time_us":1467,"apply.run_wall_time_us":1467,"num_ops":1,"prepare.queue_time_us":12004,"prepare.run_cpu_time_us":3765,"prepare.run_wall_time_us":156071,"raft.queue_time_us":836,"raft.run_cpu_time_us":717,"raft.run_wall_time_us":717,"replication_time_us":248184,"spinlock_wait_cycles":344131456}]]}
I20260809 01:45:56.409823 12889 leader_election.cc:290] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:56.411569 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d5a709cd86476594613e788a325094" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:56.413336 10805 leader_election.cc:343] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:56.415127 10588 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fe4afb4101ec445e94210e3722725b59 consensus state has no leader
I20260809 01:45:56.425997 11239 heartbeater.cc:344] Connected to a master server at 127.10.8.125:42063
I20260809 01:45:56.438730 10986 heartbeater.cc:344] Connected to a master server at 127.10.8.125:42063
I20260809 01:45:56.453620 11551 tablet_bootstrap.cc:492] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:56.454138 11551 ts_tablet_manager.cc:1403] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.459s user 0.007s sys 0.008s
I20260809 01:45:56.457222 13048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: Bootstrap starting.
I20260809 01:45:56.459120 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b31998b65e4264b16c0d73dbcfc73b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:56.466671 12874 ts_tablet_manager.cc:1929] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:56.467092 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325e3d19b83545a197c2e09fc9643b5e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:56.473255 11062 consensus_peers.cc:597] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:56.474143 11553 tablet_bootstrap.cc:492] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:56.474733 11553 ts_tablet_manager.cc:1403] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.303s user 0.031s sys 0.036s
I20260809 01:45:56.475054 12874 log.cc:1199] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/75e4315b3c644846ac572883c51f23de
W20260809 01:45:56.475641 10804 leader_election.cc:343] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:56.475929 12874 ts_tablet_manager.cc:1950] T 75e4315b3c644846ac572883c51f23de P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:56.476044 10804 leader_election.cc:304] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:56.476970 12889 raft_consensus.cc:2749] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:56.468143 11554 tablet_bootstrap.cc:492] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:56.477502 11554 ts_tablet_manager.cc:1403] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.238s user 0.025s sys 0.028s
W20260809 01:45:56.478019 10804 leader_election.cc:343] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:56.477952 11553 raft_consensus.cc:359] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:56.478386 10804 leader_election.cc:304] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:56.478646 11553 raft_consensus.cc:385] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:56.478968 11553 raft_consensus.cc:740] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:56.479365 12889 raft_consensus.cc:2749] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:56.479578 11553 consensus_queue.cc:260] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:56.483894 11554 raft_consensus.cc:359] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.484472 11554 raft_consensus.cc:385] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:56.484719 11554 raft_consensus.cc:740] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:56.485528 12982 raft_consensus.cc:493] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:56.485955 12982 raft_consensus.cc:515] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.487928 12982 leader_election.cc:290] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:56.488680 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7803a6aff02d42c1833f9b2cd7332b9f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:56.488749 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d5a709cd86476594613e788a325094" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:56.489326 11156 raft_consensus.cc:2468] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
W20260809 01:45:56.490002 11289 leader_election.cc:343] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 7803a6aff02d42c1833f9b2cd7332b9f
I20260809 01:45:56.490598 11283 leader_election.cc:304] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:56.491775 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 75e4315b3c644846ac572883c51f23de (table default.Table_ee7c1665ed2344278d1ee031c67438a4 [id=850f362a7f9a4531a3d7d6de29b695ba]) successfully deleted
I20260809 01:45:56.492184 12982 raft_consensus.cc:2804] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:56.492481 12982 raft_consensus.cc:493] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:56.492655 11554 consensus_queue.cc:260] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.493129 12982 raft_consensus.cc:3060] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:56.494874 11554 ts_tablet_manager.cc:1434] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.017s user 0.004s sys 0.000s
I20260809 01:45:56.494820 12889 raft_consensus.cc:493] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:56.495247 12889 raft_consensus.cc:515] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.498387 12889 leader_election.cc:290] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:56.509816 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c704d148da8d406aa824253a51b850c8" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:56.512256 10805 leader_election.cc:343] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:56.513556 11554 tablet_bootstrap.cc:492] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
W20260809 01:45:56.526808 11063 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.10.8.124:33929, user_credentials={real_user=slave}} blocked reactor thread for 44996us
W20260809 01:45:56.529084 11063 leader_election.cc:343] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 91941616dfdd4df3aaa7c8460876d273
I20260809 01:45:56.529510 11063 leader_election.cc:304] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:56.530475 12778 raft_consensus.cc:2749] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:56.533617 13028 consensus_queue.cc:1048] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:56.535765 12982 raft_consensus.cc:515] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:56.536847 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet efe4c7b15604442eb7be83fe19bb54f4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet efe4c7b15604442eb7be83fe19bb54f4 already in progress: creating tablet
I20260809 01:45:56.537130 11465 heartbeater.cc:344] Connected to a master server at 127.10.8.125:42063
W20260809 01:45:56.537496 10579 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 44948.1us
I20260809 01:45:56.538568 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7803a6aff02d42c1833f9b2cd7332b9f" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
W20260809 01:45:56.538801 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 6b94950efa2a49588fc2a69082e33acb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6b94950efa2a49588fc2a69082e33acb already in progress: creating tablet
I20260809 01:45:56.539026 11553 ts_tablet_manager.cc:1434] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.064s user 0.008s sys 0.005s
I20260809 01:45:56.539045 11156 raft_consensus.cc:3060] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:56.539415 10588 master_service.cc:960] ReplaceTablet: received request to replace tablet f7d7a8f179244b3b9c9ca042af67e4c0 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:56.539971 11553 tablet_bootstrap.cc:492] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:56.545432 11156 raft_consensus.cc:2468] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:56.551868 11553 tablet_bootstrap.cc:654] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:56.552376 11283 leader_election.cc:304] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:45:56.561761 11554 tablet_bootstrap.cc:654] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:56.562598 12653 raft_consensus.cc:2804] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
W20260809 01:45:56.565805 11289 leader_election.cc:343] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 7803a6aff02d42c1833f9b2cd7332b9f
I20260809 01:45:56.565905 13048 log.cc:826] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: Log is configured to *not* fsync() on all Append() calls
I20260809 01:45:56.566460 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_45e8dad480984c1ca35198bd7cd125c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:56.568039 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_45e8dad480984c1ca35198bd7cd125c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:56.572043 12982 leader_election.cc:290] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:56.576807 13028 raft_consensus.cc:493] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:56.577456 13028 raft_consensus.cc:515] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.581035 10591 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_0bcdcc37d5f04f40ab4dd02b66baf4a6" } modify_external_catalogs: true
I20260809 01:45:56.581508 10591 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_0bcdcc37d5f04f40ab4dd02b66baf4a6" } modify_external_catalogs: true
I20260809 01:45:56.583338 13028 leader_election.cc:290] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:56.595999 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2692a931500443d298e25a2b362b22a1" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:56.597885 11062 leader_election.cc:343] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:56.456956 11551 raft_consensus.cc:359] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.608199 11551 raft_consensus.cc:385] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:56.608520 11551 raft_consensus.cc:740] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:56.609508 11551 consensus_queue.cc:260] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:56.624574 10596 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 68e6698847804951a7cb2db0ae1bbf94 consensus state has no leader
I20260809 01:45:56.625766 13053 raft_consensus.cc:493] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:56.626994 10596 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_836b8d3fc2e34cb786cadfc1794cb1ad" } new_table_name: "default.Table_17ac5d26975e4c1592d04b6573a6758d" modify_external_catalogs: true
I20260809 01:45:56.626565 13053 raft_consensus.cc:515] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:56.631561 13053 leader_election.cc:290] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:56.632797 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad9cdaa5c0a4dc7bfc34db8f18c2b6b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:56.634641 10805 leader_election.cc:343] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:56.656963 12904 ts_tablet_manager.cc:1929] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:56.657212 13048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260809 01:45:56.659392 13048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: Bootstrap complete.
I20260809 01:45:56.664428 12904 log.cc:1199] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/75e4315b3c644846ac572883c51f23de
I20260809 01:45:56.697593 13028 raft_consensus.cc:493] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:56.698253 13028 raft_consensus.cc:515] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.700650 13028 leader_election.cc:290] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:56.703989 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3bf43978dc94c2b9a7f3c18a4677c40" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:56.706971 11062 leader_election.cc:343] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:56.714941 12904 ts_tablet_manager.cc:1950] T 75e4315b3c644846ac572883c51f23de P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
W20260809 01:45:56.718860 11063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 189783us
W20260809 01:45:56.722865 10804 leader_election.cc:343] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:56.723337 10804 leader_election.cc:304] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:56.724279 13053 raft_consensus.cc:2749] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:56.719732 13048 raft_consensus.cc:359] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:56.734864 11551 ts_tablet_manager.cc:1434] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.280s user 0.005s sys 0.000s
I20260809 01:45:56.736142 11551 tablet_bootstrap.cc:492] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:56.740254 11554 tablet_bootstrap.cc:492] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:56.740796 11554 ts_tablet_manager.cc:1403] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.227s user 0.013s sys 0.027s
I20260809 01:45:56.745666 12653 raft_consensus.cc:697] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:56.747370 12653 consensus_queue.cc:237] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.750047 13048 raft_consensus.cc:740] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bc72deb1d2402cabd817dfe3b27e88, State: Initialized, Role: FOLLOWER
I20260809 01:45:56.751849 13048 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:56.754513 13056 raft_consensus.cc:493] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:56.766496 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2692a931500443d298e25a2b362b22a1" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:56.767016 10905 raft_consensus.cc:2468] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:56.769414 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c704d148da8d406aa824253a51b850c8" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:56.771683 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad9cdaa5c0a4dc7bfc34db8f18c2b6b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:56.788328 10576 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 8baa4ae62edf46b5a386661a6826bbab (table default.Table_fba7d62f160e436c9d150c884a5e013a [id=b15bb9c641894a39a13c4409f4c51545], current schema version=4) failed: Not found: failed to reset TS proxy: No leader found: Tablet 8baa4ae62edf46b5a386661a6826bbab consensus state has no leader
I20260809 01:45:56.744946 11554 raft_consensus.cc:359] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:56.788645 11554 raft_consensus.cc:385] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:56.788946 11554 raft_consensus.cc:740] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:56.790012 12904 tablet_replica.cc:333] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:56.790819 12904 raft_consensus.cc:2243] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:56.791289 12904 raft_consensus.cc:2272] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:56.790971 13066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:45:56.791874 13066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:45:56.794952 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 75e4315b3c644846ac572883c51f23de (table default.Table_ee7c1665ed2344278d1ee031c67438a4 [id=850f362a7f9a4531a3d7d6de29b695ba]) successfully deleted
I20260809 01:45:56.795233 11554 consensus_queue.cc:260] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:56.795676 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 6b94950efa2a49588fc2a69082e33acb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6b94950efa2a49588fc2a69082e33acb already in progress: creating tablet
W20260809 01:45:56.796379 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet efe4c7b15604442eb7be83fe19bb54f4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet efe4c7b15604442eb7be83fe19bb54f4 already in progress: creating tablet
I20260809 01:45:56.797264 11554 ts_tablet_manager.cc:1434] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.056s user 0.005s sys 0.004s
I20260809 01:45:56.798223 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_190f8292dbc54632996f644b10d5c284"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:56.799834 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_190f8292dbc54632996f644b10d5c284 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:56.799999 10804 leader_election.cc:343] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:56.800351 10804 leader_election.cc:304] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:56.801195 13053 raft_consensus.cc:2749] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:56.808807 11063 leader_election.cc:304] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:56.811141 13028 raft_consensus.cc:2804] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:56.811460 13028 raft_consensus.cc:493] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:56.811837 13028 raft_consensus.cc:3060] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:56.833179 12778 raft_consensus.cc:493] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:56.833659 12778 raft_consensus.cc:515] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:56.835701 11063 leader_election.cc:343] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: c3bf43978dc94c2b9a7f3c18a4677c40
I20260809 01:45:56.836202 11554 tablet_bootstrap.cc:492] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:56.836149 11063 leader_election.cc:304] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:56.836663 12778 leader_election.cc:290] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:56.838019 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9493892c1ff48e7a0558ff0fcb7a0f3" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:56.839645 11062 leader_election.cc:343] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:56.840013 11553 tablet_bootstrap.cc:492] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:56.840468 11553 ts_tablet_manager.cc:1403] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.301s user 0.018s sys 0.023s
I20260809 01:45:56.842233 13053 raft_consensus.cc:493] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:56.843047 13053 raft_consensus.cc:515] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.845423 11341 tablet_service.cc:1558] Processing DeleteTablet for tablet 75e4315b3c644846ac572883c51f23de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:55 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:56.845181 11553 raft_consensus.cc:359] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.846153 11553 raft_consensus.cc:385] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:56.846719 13053 leader_election.cc:290] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:56.847298 11341 tablet_service.cc:1558] Processing DeleteTablet for tablet 995a37a9e6b14889b876da5d76c89b84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:39 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:56.847854 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc9c8e0f97e41b48a919aa579ad85cd" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:56.848194 10804 leader_election.cc:343] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:56.848605 10804 leader_election.cc:304] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:56.848959 11341 tablet_service.cc:1558] Processing DeleteTablet for tablet 36a542c773b7438ead16c5b3af224cf9 with delete_type TABLET_DATA_DELETED (replaced by tablet a38bd344108649ed8e1687f82154000b) from {username='slave'} at 127.0.0.1:53476
W20260809 01:45:56.849261 10805 leader_election.cc:343] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:56.849581 13053 raft_consensus.cc:2749] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:56.846442 11553 raft_consensus.cc:740] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:56.851058 11553 consensus_queue.cc:260] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.852293 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 197b5043f6d348f89853c073b256daa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:56.853664 11341 tablet_service.cc:1558] Processing DeleteTablet for tablet b84f1943c1a34e999fc7c12a5847cdb6 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ed45a2d5e9b4ad8be6f94e4a895003f) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:56.856825 12874 tablet_replica.cc:333] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:56.755291 13056 raft_consensus.cc:515] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.858407 12874 raft_consensus.cc:2243] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:56.793103 12904 ts_tablet_manager.cc:1916] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:56.859243 12874 raft_consensus.cc:2272] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:56.854679 11553 ts_tablet_manager.cc:1434] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.014s user 0.006s sys 0.004s
I20260809 01:45:56.866649 11553 tablet_bootstrap.cc:492] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:56.870648 11551 tablet_bootstrap.cc:654] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:56.871146 12874 ts_tablet_manager.cc:1916] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:56.873776 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 75e4315b3c644846ac572883c51f23de because the tablet was not found. No further retry: Not found: Tablet not found: 75e4315b3c644846ac572883c51f23de
I20260809 01:45:56.882560 12653 raft_consensus.cc:493] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:56.883458 12653 raft_consensus.cc:515] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:56.843916 10592 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.67:40463 (request call id 58) took 924 ms (client timeout 1000 ms). Trace:
W20260809 01:45:56.885986 10592 rpcz_store.cc:269] 0809 01:45:55.919191 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:55.919295 (+ 104us) service_pool.cc:224] Handling call
0809 01:45:56.078209 (+158914us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:56.843805 (+765596us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:56.758144 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:56.758665 (+ 521us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:56.758728 (+ 63us) write_op.cc:435] Acquired schema lock
0809 01:45:56.758741 (+ 13us) tablet.cc:662] Decoding operations
0809 01:45:56.759102 (+ 361us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:56.759140 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:56.759173 (+ 33us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:56.759400 (+ 227us) tablet.cc:701] Row locks acquired
0809 01:45:56.759413 (+ 13us) write_op.cc:260] PREPARE: finished
0809 01:45:56.759544 (+ 131us) write_op.cc:270] Start()
0809 01:45:56.759650 (+ 106us) write_op.cc:276] Timestamp: P: 1786239956759482 usec, L: 0
0809 01:45:56.759676 (+ 26us) op_driver.cc:348] REPLICATION: starting
0809 01:45:56.760338 (+ 662us) log.cc:844] Serialized 527 byte log entry
0809 01:45:56.819289 (+ 58951us) op_driver.cc:464] REPLICATION: finished
0809 01:45:56.825414 (+ 6125us) write_op.cc:301] APPLY: starting
0809 01:45:56.825535 (+ 121us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:56.825694 (+ 159us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:56.825927 (+ 233us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:56.826267 (+ 340us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:56.826558 (+ 291us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:56.826651 (+ 93us) write_op.cc:312] APPLY: finished
0809 01:45:56.826953 (+ 302us) log.cc:844] Serialized 23 byte log entry
0809 01:45:56.827150 (+ 197us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:56.827251 (+ 101us) write_op.cc:454] Released schema lock
0809 01:45:56.827313 (+ 62us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":11,"spinlock_wait_cycles":359385856,"child_traces":[["op",{"apply.queue_time_us":5894,"num_ops":1,"prepare.queue_time_us":596275,"prepare.run_cpu_time_us":2464,"prepare.run_wall_time_us":2465,"replication_time_us":59484}]]}
I20260809 01:45:56.889700 10903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3684fa56f9549f492a96a5c765eda78" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:56.891292 12653 leader_election.cc:290] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:56.891754 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3684fa56f9549f492a96a5c765eda78" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:56.894218 12599 raft_consensus.cc:2749] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:56.895299 11063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 59608.7us
I20260809 01:45:56.902316 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 7706838afc9f4dbebc3213b82c4da126
I20260809 01:45:56.905332 13048 sys_catalog.cc:565] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:45:56.908588 11553 tablet_bootstrap.cc:654] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:56.905872 11289 leader_election.cc:304] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:56.910210 12982 raft_consensus.cc:2749] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:56.908644 11554 tablet_bootstrap.cc:654] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:56.918028 11283 leader_election.cc:343] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: b2726cb09f734ed18bb9379f94f3f22a
I20260809 01:45:56.923615 11551 tablet_bootstrap.cc:492] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:56.924113 11551 ts_tablet_manager.cc:1403] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.188s user 0.007s sys 0.005s
I20260809 01:45:56.955515 11551 raft_consensus.cc:359] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:56.956360 11551 raft_consensus.cc:385] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:56.956660 11551 raft_consensus.cc:740] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:56.956786 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc9c8e0f97e41b48a919aa579ad85cd" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:56.957396 11551 consensus_queue.cc:260] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:56.958678 10804 leader_election.cc:343] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:56.965885 10804 leader_election.cc:304] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:56.966950 13053 raft_consensus.cc:2749] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:56.970232 13053 raft_consensus.cc:493] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:56.997751 10597 catalog_manager.cc:6807] ReplaceTablet: tablet 7706838afc9f4dbebc3213b82c4da126 deleted and replaced by tablet cec52957d4f5460d9f0846bc976926db
I20260809 01:45:57.000088 11341 tablet_service.cc:1558] Processing DeleteTablet for tablet 7706838afc9f4dbebc3213b82c4da126 with delete_type TABLET_DATA_DELETED (replaced by tablet cec52957d4f5460d9f0846bc976926db) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:57.008708 10588 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet f7d7a8f179244b3b9c9ca042af67e4c0
I20260809 01:45:57.010223 10588 catalog_manager.cc:6807] ReplaceTablet: tablet f7d7a8f179244b3b9c9ca042af67e4c0 deleted and replaced by tablet 033eaf96ef5d4c6f87c28f70bde94ff0
I20260809 01:45:57.011044 10588 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_b363ce92e78e4e45b8aa8562d738bdbd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:57.012507 10588 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b363ce92e78e4e45b8aa8562d738bdbd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:56.960312 11551 ts_tablet_manager.cc:1434] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.036s user 0.001s sys 0.005s
I20260809 01:45:56.993677 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet 7706838afc9f4dbebc3213b82c4da126 with delete_type TABLET_DATA_DELETED (replaced by tablet cec52957d4f5460d9f0846bc976926db) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:57.010625 11105 tablet_service.cc:1558] Processing DeleteTablet for tablet f7d7a8f179244b3b9c9ca042af67e4c0 with delete_type TABLET_DATA_DELETED (replaced by tablet 033eaf96ef5d4c6f87c28f70bde94ff0) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:57.016741 11551 tablet_bootstrap.cc:492] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:57.028340 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_cf7bd2b9aec04c73b8ed72debcab795d" } new_table_name: "default.Table_66160146b5684111a0b61024efd70dfb" modify_external_catalogs: true
I20260809 01:45:57.036717 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_483ea515938a4f9aa7347c3a8f948cf5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:57.039186 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_483ea515938a4f9aa7347c3a8f948cf5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:57.066425 10596 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 033eaf96ef5d4c6f87c28f70bde94ff0 has no consensus state
I20260809 01:45:57.083614 12874 ts_tablet_manager.cc:1929] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:57.084105 12874 log.cc:1199] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/995a37a9e6b14889b876da5d76c89b84
I20260809 01:45:57.084879 12874 ts_tablet_manager.cc:1950] T 995a37a9e6b14889b876da5d76c89b84 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:57.098930 12904 ts_tablet_manager.cc:1929] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:57.099526 12904 log.cc:1199] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/7803a6aff02d42c1833f9b2cd7332b9f
W20260809 01:45:57.103483 11063 leader_election.cc:343] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: b9493892c1ff48e7a0558ff0fcb7a0f3
I20260809 01:45:57.114809 11551 tablet_bootstrap.cc:654] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:57.117576 11289 leader_election.cc:343] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: b2726cb09f734ed18bb9379f94f3f22a
I20260809 01:45:57.118160 11289 leader_election.cc:304] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:57.121014 12982 raft_consensus.cc:2749] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:57.132972 12904 ts_tablet_manager.cc:1950] T 7803a6aff02d42c1833f9b2cd7332b9f P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:57.135218 13084 raft_consensus.cc:493] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:57.135861 13084 raft_consensus.cc:515] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.101622 13053 raft_consensus.cc:515] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.141094 13053 leader_election.cc:290] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:57.142601 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e360e79820d148f4a6e9fdeb2c19da54" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:57.144336 10805 leader_election.cc:343] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.144938 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8baa4ae62edf46b5a386661a6826bbab" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:57.146001 13083 raft_consensus.cc:493] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:57.146965 10805 leader_election.cc:343] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.146801 13083 raft_consensus.cc:515] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:57.150449 13084 leader_election.cc:290] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:57.156555 12904 tablet_replica.cc:333] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:57.165273 13083 leader_election.cc:290] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:57.165442 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04881d1c5254053bef50dd3d9d475f2" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:57.166973 11553 tablet_bootstrap.cc:492] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
W20260809 01:45:57.167078 10805 leader_election.cc:343] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.167436 11553 ts_tablet_manager.cc:1403] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.301s user 0.004s sys 0.020s
I20260809 01:45:57.173962 10858 tablet_service.cc:1558] Processing DeleteTablet for tablet 7706838afc9f4dbebc3213b82c4da126 with delete_type TABLET_DATA_DELETED (replaced by tablet cec52957d4f5460d9f0846bc976926db) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:57.175119 10862 tablet_service.cc:1558] Processing DeleteTablet for tablet f7d7a8f179244b3b9c9ca042af67e4c0 with delete_type TABLET_DATA_DELETED (replaced by tablet 033eaf96ef5d4c6f87c28f70bde94ff0) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:57.184096 12874 tablet_replica.cc:333] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:57.187991 12904 raft_consensus.cc:2243] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:57.195997 11553 raft_consensus.cc:359] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.135272 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 7803a6aff02d42c1833f9b2cd7332b9f (table default.Table_8a405d19ab94416c8d58fff878edae0a [id=42039f2e75f942ac8b96fb47088ad193]) successfully deleted
I20260809 01:45:57.196700 11553 raft_consensus.cc:385] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:57.196951 11553 raft_consensus.cc:740] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
W20260809 01:45:57.197105 10576 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 61870.1us
I20260809 01:45:57.197954 13056 leader_election.cc:290] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:57.198143 12874 raft_consensus.cc:2243] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:57.197110 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 995a37a9e6b14889b876da5d76c89b84 (table default.Table_50f4db524f554499a2dab6d3bc48f053 [id=85ee30d1d77d47f691e6a0dd498d9bca]) successfully deleted
I20260809 01:45:57.197767 11553 consensus_queue.cc:260] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:57.199740 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 36a542c773b7438ead16c5b3af224cf9 because the tablet was not found. No further retry: Not found: Tablet not found: 36a542c773b7438ead16c5b3af224cf9
I20260809 01:45:57.201440 11553 ts_tablet_manager.cc:1434] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.034s user 0.006s sys 0.001s
I20260809 01:45:57.185598 11341 tablet_service.cc:1558] Processing DeleteTablet for tablet f7d7a8f179244b3b9c9ca042af67e4c0 with delete_type TABLET_DATA_DELETED (replaced by tablet 033eaf96ef5d4c6f87c28f70bde94ff0) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:57.202422 11553 tablet_bootstrap.cc:492] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:57.191521 13028 raft_consensus.cc:515] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.104060 11063 leader_election.cc:304] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:57.203297 11063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 99882.5us
I20260809 01:45:57.204308 12599 raft_consensus.cc:2749] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:57.207312 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e360e79820d148f4a6e9fdeb2c19da54" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:57.208897 13028 leader_election.cc:290] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:57.209982 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8baa4ae62edf46b5a386661a6826bbab" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:57.209928 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04881d1c5254053bef50dd3d9d475f2" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:57.212414 12904 raft_consensus.cc:2272] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:57.212813 10804 consensus_peers.cc:597] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
W20260809 01:45:57.213390 10804 leader_election.cc:343] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.213786 10804 leader_election.cc:304] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:57.214147 10903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2692a931500443d298e25a2b362b22a1" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:57.214917 10903 raft_consensus.cc:3060] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:57.215097 13083 raft_consensus.cc:2749] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:57.216708 12904 ts_tablet_manager.cc:1916] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:57.218334 10804 leader_election.cc:343] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.218847 10804 leader_election.cc:304] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:57.220451 10804 leader_election.cc:343] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.220855 10804 leader_election.cc:304] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:57.219748 13083 raft_consensus.cc:2749] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:57.229388 13085 raft_consensus.cc:2749] T 8baa4ae62edf46b5a386661a6826bbab P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:57.236074 11551 tablet_bootstrap.cc:492] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:57.236694 11551 ts_tablet_manager.cc:1403] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.220s user 0.008s sys 0.008s
I20260809 01:45:57.241663 11551 raft_consensus.cc:359] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:57.254233 13086 tablet_replica.cc:333] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:57.256202 13086 raft_consensus.cc:2243] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:57.257634 11551 raft_consensus.cc:385] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:57.259017 11551 raft_consensus.cc:740] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:57.259907 11551 consensus_queue.cc:260] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:57.271740 10903 raft_consensus.cc:2468] T 2692a931500443d298e25a2b362b22a1 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
W20260809 01:45:57.274175 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 325e3d19b83545a197c2e09fc9643b5e consensus state has no leader
I20260809 01:45:57.272138 13086 raft_consensus.cc:2272] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:57.275218 11551 ts_tablet_manager.cc:1434] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.038s user 0.004s sys 0.004s
I20260809 01:45:57.278556 13086 ts_tablet_manager.cc:1916] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:57.278216 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_985df37273284bb7be1a28211209ecfa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:57.279966 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_985df37273284bb7be1a28211209ecfa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:57.279973 11063 leader_election.cc:304] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:57.280849 13028 raft_consensus.cc:2804] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:57.281678 13028 raft_consensus.cc:697] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:57.283128 13028 consensus_queue.cc:237] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.281026 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_49d42634313a4792b68ba9d39e4219f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:57.286809 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49d42634313a4792b68ba9d39e4219f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:57.294073 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_a86a2cb7b56442e0aed85bbde8eba334" } new_table_name: "default.Table_ca5dbb738ded4d3494afe46eb820bea1" modify_external_catalogs: true
I20260809 01:45:57.289009 11553 tablet_bootstrap.cc:654] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:57.320104 13093 raft_consensus.cc:493] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:57.327947 11553 tablet_bootstrap.cc:492] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:57.328389 11553 ts_tablet_manager.cc:1403] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.126s user 0.003s sys 0.014s
I20260809 01:45:57.328675 13093 raft_consensus.cc:515] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:57.332504 11553 raft_consensus.cc:359] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.333293 11553 raft_consensus.cc:385] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:57.336210 11553 raft_consensus.cc:740] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:57.336907 11553 consensus_queue.cc:260] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.354863 11553 ts_tablet_manager.cc:1434] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.026s user 0.006s sys 0.002s
I20260809 01:45:57.357416 11553 tablet_bootstrap.cc:492] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:57.365988 12904 ts_tablet_manager.cc:1929] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:57.366427 12904 log.cc:1199] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/7706838afc9f4dbebc3213b82c4da126
I20260809 01:45:57.367262 12904 ts_tablet_manager.cc:1950] T 7706838afc9f4dbebc3213b82c4da126 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:57.398754 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2692a931500443d298e25a2b362b22a1" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:57.199031 12874 raft_consensus.cc:2272] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:57.404678 12874 ts_tablet_manager.cc:1916] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:57.412192 11062 leader_election.cc:343] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.415614 11554 tablet_bootstrap.cc:492] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:57.416033 11554 ts_tablet_manager.cc:1403] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.580s user 0.016s sys 0.013s
I20260809 01:45:57.420892 11554 raft_consensus.cc:359] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.422885 11554 raft_consensus.cc:385] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:57.423213 11554 raft_consensus.cc:740] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:57.438851 11551 tablet_bootstrap.cc:492] T 33a5ef5a3ff44f74ab19998ba9f19310 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:57.449930 11554 consensus_queue.cc:260] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.452064 11554 ts_tablet_manager.cc:1434] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.036s user 0.005s sys 0.004s
I20260809 01:45:57.453060 11554 tablet_bootstrap.cc:492] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:57.441624 13093 leader_election.cc:290] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:57.451802 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]) successfully deleted
W20260809 01:45:57.471807 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet f7d7a8f179244b3b9c9ca042af67e4c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f7d7a8f179244b3b9c9ca042af67e4c0 already in progress: creating tablet
I20260809 01:45:57.483237 11551 tablet_bootstrap.cc:654] T 33a5ef5a3ff44f74ab19998ba9f19310 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:57.496236 10598 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_13127c445be045458496dece32356891" } new_table_name: "default.Table_81380e3f093c48cb8811640969271318" modify_external_catalogs: true
I20260809 01:45:57.497964 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:45:57.506116 13083 raft_consensus.cc:493] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:57.506556 13083 raft_consensus.cc:515] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.507462 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_148f7d52dc6c4c329484c2fc8b366693"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:57.508901 13083 leader_election.cc:290] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:57.509018 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_148f7d52dc6c4c329484c2fc8b366693 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:57.511152 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6c244108a4ff3a2df0585e5280294" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:57.512439 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6c244108a4ff3a2df0585e5280294" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:57.516213 13085 raft_consensus.cc:493] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:57.516947 10805 leader_election.cc:343] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:57.517786 10804 leader_election.cc:343] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.518148 10804 leader_election.cc:304] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:57.521463 11554 tablet_bootstrap.cc:654] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:57.516639 13085 raft_consensus.cc:515] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.550246 11395 raft_consensus.cc:3060] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:57.558521 12874 ts_tablet_manager.cc:1929] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260809 01:45:57.557986 10598 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b5bd08670e2e40e0bb250d5e054a4116 has no consensus state
I20260809 01:45:57.571712 12874 log.cc:1199] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/197b5043f6d348f89853c073b256daa4
I20260809 01:45:57.573262 12874 ts_tablet_manager.cc:1950] T 197b5043f6d348f89853c073b256daa4 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:57.579146 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:45:57.584556 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_2708bfe75ad04882bc9a84d8e502c837"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:57.586589 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2708bfe75ad04882bc9a84d8e502c837 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:57.588290 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_a2532aa2811644d3ae76f4cdc35c0518"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:57.589378 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 197b5043f6d348f89853c073b256daa4 (table default.Table_5f5f7b17133f4f4d9cfcac1aa0d270fa [id=12a77217eec442c6a9199a418d9d5bb8]) successfully deleted
W20260809 01:45:57.589996 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2532aa2811644d3ae76f4cdc35c0518 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:57.593047 10595 master_service.cc:960] ReplaceTablet: received request to replace tablet b6159d660d544b3e907a08372027c0e2 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:57.594143 12870 leader_election.cc:304] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bc72deb1d2402cabd817dfe3b27e88; no voters: 09a73ee8e8ec496a984efcab27715506, 629f002048f64491afdd57f72a466e59
I20260809 01:45:57.585039 13084 raft_consensus.cc:2749] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:57.596089 13093 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 01:45:57.598138 13085 leader_election.cc:290] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:45:57.598858 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet b84f1943c1a34e999fc7c12a5847cdb6 because the tablet was not found. No further retry: Not found: Tablet not found: b84f1943c1a34e999fc7c12a5847cdb6
I20260809 01:45:57.599202 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058dfc9b37f04fd8a72981b4290390b4" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:57.600390 12874 tablet_replica.cc:333] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
W20260809 01:45:57.600731 10805 leader_election.cc:343] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.602393 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058dfc9b37f04fd8a72981b4290390b4" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:57.604475 10804 leader_election.cc:343] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.604861 10804 leader_election.cc:304] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:57.606235 13083 raft_consensus.cc:2749] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:57.607432 11551 tablet_bootstrap.cc:492] T 33a5ef5a3ff44f74ab19998ba9f19310 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:57.607923 11551 ts_tablet_manager.cc:1403] T 33a5ef5a3ff44f74ab19998ba9f19310 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.169s user 0.017s sys 0.026s
I20260809 01:45:57.609169 11554 tablet_bootstrap.cc:492] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:57.609632 11554 ts_tablet_manager.cc:1403] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.157s user 0.008s sys 0.010s
I20260809 01:45:57.610996 11551 raft_consensus.cc:359] T 33a5ef5a3ff44f74ab19998ba9f19310 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.611748 11551 raft_consensus.cc:385] T 33a5ef5a3ff44f74ab19998ba9f19310 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:57.612062 11551 raft_consensus.cc:740] T 33a5ef5a3ff44f74ab19998ba9f19310 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:57.612751 12874 raft_consensus.cc:2243] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:57.613857 12874 raft_consensus.cc:2272] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:57.618498 11553 tablet_bootstrap.cc:654] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:57.620812 12874 ts_tablet_manager.cc:1916] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:57.621101 11551 consensus_queue.cc:260] T 33a5ef5a3ff44f74ab19998ba9f19310 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.537493 13086 ts_tablet_manager.cc:1929] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:57.625202 13086 log.cc:1199] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/7706838afc9f4dbebc3213b82c4da126
I20260809 01:45:57.626430 13086 ts_tablet_manager.cc:1950] T 7706838afc9f4dbebc3213b82c4da126 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:57.631490 13085 raft_consensus.cc:493] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:57.632212 13085 raft_consensus.cc:515] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.634296 10588 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_5f68b1cdb1a347f78466f3333fd113ff" } new_table_name: "default.Table_bbf8548d28c748e19d427c4f7f79622f" modify_external_catalogs: true
I20260809 01:45:57.636004 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e2f77a70794ff7811b5f09d885904b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:57.637178 10804 leader_election.cc:343] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.612053 11554 raft_consensus.cc:359] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.637653 11554 raft_consensus.cc:385] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:57.637933 11554 raft_consensus.cc:740] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:57.638276 13085 leader_election.cc:290] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:57.639462 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e2f77a70794ff7811b5f09d885904b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:57.640780 10805 leader_election.cc:343] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.641165 10805 leader_election.cc:304] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:57.642391 11551 ts_tablet_manager.cc:1434] T 33a5ef5a3ff44f74ab19998ba9f19310 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.034s user 0.007s sys 0.000s
I20260809 01:45:57.642894 11554 consensus_queue.cc:260] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.647505 11554 ts_tablet_manager.cc:1434] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.038s user 0.005s sys 0.000s
I20260809 01:45:57.647899 13085 raft_consensus.cc:2749] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:57.654006 11551 tablet_bootstrap.cc:492] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:57.673600 11554 tablet_bootstrap.cc:492] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:57.675499 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]) successfully deleted
W20260809 01:45:57.678352 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet f7d7a8f179244b3b9c9ca042af67e4c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f7d7a8f179244b3b9c9ca042af67e4c0 already in progress: creating tablet
I20260809 01:45:57.692183 11554 tablet_bootstrap.cc:654] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:57.698570 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_3c13b21368154865a2a3d2b18f7db9ca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:57.700222 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c13b21368154865a2a3d2b18f7db9ca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:57.701191 11553 tablet_bootstrap.cc:492] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:57.707341 11553 ts_tablet_manager.cc:1403] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.350s user 0.007s sys 0.007s
I20260809 01:45:57.716148 11553 raft_consensus.cc:359] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.718896 11551 tablet_bootstrap.cc:654] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:57.717622 13085 raft_consensus.cc:493] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:57.733028 13085 raft_consensus.cc:515] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.734287 12874 ts_tablet_manager.cc:1929] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:57.734697 12874 log.cc:1199] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/7706838afc9f4dbebc3213b82c4da126
I20260809 01:45:57.736032 12874 ts_tablet_manager.cc:1950] T 7706838afc9f4dbebc3213b82c4da126 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:57.736969 13085 leader_election.cc:290] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:57.738155 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a5ef5a3ff44f74ab19998ba9f19310" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:57.739024 11553 raft_consensus.cc:385] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:57.738924 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a5ef5a3ff44f74ab19998ba9f19310" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:57.739463 11156 raft_consensus.cc:2468] T 33a5ef5a3ff44f74ab19998ba9f19310 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 0.
W20260809 01:45:57.739688 10805 leader_election.cc:343] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.740402 11553 raft_consensus.cc:740] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:57.740618 10804 leader_election.cc:304] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:57.741729 13085 raft_consensus.cc:2804] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:57.742410 13085 raft_consensus.cc:493] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:57.743058 13085 raft_consensus.cc:3060] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:57.741047 11553 consensus_queue.cc:260] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.754273 11239 heartbeater.cc:344] Connected to a master server at 127.10.8.125:42063
I20260809 01:45:57.762835 11553 ts_tablet_manager.cc:1434] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.049s user 0.004s sys 0.000s
I20260809 01:45:57.763940 11553 tablet_bootstrap.cc:492] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:57.771235 11554 tablet_bootstrap.cc:492] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:57.776449 11554 ts_tablet_manager.cc:1403] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.105s user 0.005s sys 0.009s
I20260809 01:45:57.784195 11551 tablet_bootstrap.cc:492] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:57.784698 11551 ts_tablet_manager.cc:1403] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.131s user 0.006s sys 0.011s
I20260809 01:45:57.800110 13083 raft_consensus.cc:493] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:57.800838 13083 raft_consensus.cc:515] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.804911 13083 leader_election.cc:290] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:57.806809 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c87361b29664464f9fd73be974e2d3a0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:57.808516 10804 leader_election.cc:343] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.810775 11551 raft_consensus.cc:359] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.811570 11551 raft_consensus.cc:385] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:57.811872 11551 raft_consensus.cc:740] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
W20260809 01:45:57.815919 10581 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 63c1a610663a4a2fa58f6cdc5f8d86ce (table default.Table_bd6eedffdf6440efa3e776b7079015d8 [id=8a2084683a7740db80e2c52d3f4c9506], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 63c1a610663a4a2fa58f6cdc5f8d86ce consensus state has no leader
I20260809 01:45:57.816303 11553 tablet_bootstrap.cc:654] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:57.820302 11551 consensus_queue.cc:260] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.830682 13028 raft_consensus.cc:493] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:57.831624 13028 raft_consensus.cc:515] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.835983 11551 ts_tablet_manager.cc:1434] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.051s user 0.007s sys 0.002s
I20260809 01:45:57.835819 13085 raft_consensus.cc:515] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.836870 11551 tablet_bootstrap.cc:492] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
W20260809 01:45:57.837342 11063 leader_election.cc:343] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 403cb09c1786491ab0d40502038bf9c0
I20260809 01:45:57.838542 13028 leader_election.cc:290] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:57.839186 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a5ef5a3ff44f74ab19998ba9f19310" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:57.839774 11149 raft_consensus.cc:3060] T 33a5ef5a3ff44f74ab19998ba9f19310 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:57.852109 11553 tablet_bootstrap.cc:492] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:57.852562 11553 ts_tablet_manager.cc:1403] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.089s user 0.006s sys 0.007s
I20260809 01:45:57.860265 11551 tablet_bootstrap.cc:654] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:57.860395 10592 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet cfb9afd3be194830ba99f5e20fdaca0a (table default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]). Expected version 1 got 0
I20260809 01:45:57.861533 10592 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet e3684fa56f9549f492a96a5c765eda78 (table default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]). Expected version 1 got 0
I20260809 01:45:57.862709 11105 tablet_service.cc:1511] Processing CreateTablet for tablet 0b2bb9c64aa1462eb4b14f0f11614149 (DEFAULT_TABLE table=default.Table_447d9518a2a7484ba63a2950dea459df [id=34c434cfe1034680b47af02d4ded4684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:57.863744 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b2bb9c64aa1462eb4b14f0f11614149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:57.864389 11553 raft_consensus.cc:359] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.864997 11553 raft_consensus.cc:385] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:57.870622 10586 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet cfb9afd3be194830ba99f5e20fdaca0a (table default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]). Expected version 1 got 0
I20260809 01:45:57.871234 13085 leader_election.cc:290] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:57.874830 10599 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet cfb9afd3be194830ba99f5e20fdaca0a (table default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]). Expected version 1 got 0
I20260809 01:45:57.865273 11553 raft_consensus.cc:740] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:57.878588 11553 consensus_queue.cc:260] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.879153 10596 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet cfb9afd3be194830ba99f5e20fdaca0a (table default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]). Expected version 1 got 0
I20260809 01:45:57.880327 10596 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet a04881d1c5254053bef50dd3d9d475f2 (table default.Table_57234dd818414b7397d1ac621bdb06a7 [id=17cbc8bf77e142d08d03bd7a051e00ad]). Expected version 2 got 0
W20260809 01:45:57.887073 10592 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.67:40463 (request call id 59) took 989 ms (client timeout 1000 ms). Trace:
W20260809 01:45:57.887488 10592 rpcz_store.cc:269] 0809 01:45:56.898080 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:56.925896 (+ 27816us) service_pool.cc:224] Handling call
0809 01:45:56.986015 (+ 60119us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:57.887003 (+900988us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":6,"spinlock_wait_cycles":6401408}
I20260809 01:45:57.888425 13028 raft_consensus.cc:493] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:57.889217 10596 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet e3684fa56f9549f492a96a5c765eda78 (table default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]). Expected version 1 got 0
I20260809 01:45:57.889038 13028 raft_consensus.cc:515] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.890314 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet c704d148da8d406aa824253a51b850c8
I20260809 01:45:57.890429 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403cb09c1786491ab0d40502038bf9c0" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:57.892500 10589 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 421) took 1814 ms. Trace:
I20260809 01:45:57.894344 10903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b297f9083744e2953d41cd7b1055df" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:57.894893 11553 ts_tablet_manager.cc:1434] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.042s user 0.009s sys 0.000s
I20260809 01:45:57.895301 10903 raft_consensus.cc:2468] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:57.896916 10858 tablet_service.cc:1558] Processing DeleteTablet for tablet c704d148da8d406aa824253a51b850c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:56 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:57.899398 11063 leader_election.cc:304] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:57.900024 12599 raft_consensus.cc:2804] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:57.900288 12599 raft_consensus.cc:493] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:57.900547 12599 raft_consensus.cc:3060] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:57.900483 13028 leader_election.cc:290] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:57.901497 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_ff631a5d638e4a32a994369553091330"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:57.902982 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff631a5d638e4a32a994369553091330 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:57.906486 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b297f9083744e2953d41cd7b1055df" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:57.908365 11062 leader_election.cc:343] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:57.909507 11062 leader_election.cc:343] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.909861 11062 leader_election.cc:304] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:57.909893 11554 raft_consensus.cc:359] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.910485 11554 raft_consensus.cc:385] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:57.910878 13028 raft_consensus.cc:2749] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:57.910740 11554 raft_consensus.cc:740] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:57.911835 11554 consensus_queue.cc:260] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.913849 11554 ts_tablet_manager.cc:1434] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.137s user 0.005s sys 0.003s
I20260809 01:45:57.915000 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet c704d148da8d406aa824253a51b850c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:56 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:57.914856 11554 tablet_bootstrap.cc:492] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:57.917727 10858 tablet_service.cc:1511] Processing CreateTablet for tablet 0b2bb9c64aa1462eb4b14f0f11614149 (DEFAULT_TABLE table=default.Table_447d9518a2a7484ba63a2950dea459df [id=34c434cfe1034680b47af02d4ded4684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:57.917896 12599 raft_consensus.cc:515] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.919525 13085 raft_consensus.cc:493] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:57.919979 13085 raft_consensus.cc:515] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.816728 13111 raft_consensus.cc:493] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:57.921797 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet c704d148da8d406aa824253a51b850c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c704d148da8d406aa824253a51b850c8 already in progress: creating tablet
I20260809 01:45:57.923544 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b297f9083744e2953d41cd7b1055df" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:57.922935 13111 raft_consensus.cc:515] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:45:57.925084 11062 leader_election.cc:343] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.926244 12599 leader_election.cc:290] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:57.892818 10589 rpcz_store.cc:276] 0809 01:45:56.078011 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:56.173291 (+ 95280us) service_pool.cc:224] Handling call
0809 01:45:56.189839 (+ 16548us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c86954d6 00007fa2c624db34 00007fa2c624dac2 00007fa2c624d9f1 00007fa2c624d96c 00007fa2c624cc22 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:56.190890 (+ 1051us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:56.191963 (+ 1073us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:56.192117 (+ 154us) catalog_manager.cc:2850] Locking tablets
0809 01:45:56.193991 (+ 1874us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:56.903055 (+709064us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:56.981039 (+ 77984us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:57.890206 (+909167us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:57.892400 (+ 2194us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:56.760767 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:56.761360 (+ 593us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:56.761398 (+ 38us) write_op.cc:435] Acquired schema lock
0809 01:45:56.761423 (+ 25us) tablet.cc:662] Decoding operations
0809 01:45:56.817046 (+ 55623us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:56.817082 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:56.817102 (+ 20us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:56.817348 (+ 246us) tablet.cc:701] Row locks acquired
0809 01:45:56.817360 (+ 12us) write_op.cc:260] PREPARE: finished
0809 01:45:56.817459 (+ 99us) write_op.cc:270] Start()
0809 01:45:56.817827 (+ 368us) write_op.cc:276] Timestamp: P: 1786239956817398 usec, L: 0
0809 01:45:56.817846 (+ 19us) op_driver.cc:348] REPLICATION: starting
0809 01:45:56.818453 (+ 607us) log.cc:844] Serialized 756 byte log entry
0809 01:45:56.865928 (+ 47475us) op_driver.cc:464] REPLICATION: finished
0809 01:45:56.875497 (+ 9569us) write_op.cc:301] APPLY: starting
0809 01:45:56.875615 (+ 118us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:56.875848 (+ 233us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:56.876111 (+ 263us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:56.876768 (+ 657us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:56.877096 (+ 328us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:56.877222 (+ 126us) write_op.cc:312] APPLY: finished
0809 01:45:56.901835 (+ 24613us) log.cc:844] Serialized 29 byte log entry
0809 01:45:56.901965 (+ 130us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:56.902019 (+ 54us) write_op.cc:454] Released schema lock
0809 01:45:56.902058 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:56.982620 (+ 80562us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:56.983787 (+ 1167us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":35225472,"child_traces":[["op",{"apply.queue_time_us":9240,"apply.run_cpu_time_us":2698,"apply.run_wall_time_us":105750,"num_ops":2,"prepare.queue_time_us":553696,"prepare.run_cpu_time_us":2511,"prepare.run_wall_time_us":57946,"replication_time_us":47895}]]}
I20260809 01:45:57.927318 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b297f9083744e2953d41cd7b1055df" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:57.927834 10908 raft_consensus.cc:3060] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:57.928491 11289 leader_election.cc:343] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: c528b45de0d14c19af5d6836b7a14f1c
I20260809 01:45:57.929412 13111 leader_election.cc:290] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:45:57.935887 10908 raft_consensus.cc:2468] T 33b297f9083744e2953d41cd7b1055df P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:57.938673 11063 leader_election.cc:304] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:57.938892 13085 leader_election.cc:290] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:57.939811 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c7ec3928224059ab7f58d203224536" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:57.940935 10804 leader_election.cc:343] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.943713 13028 raft_consensus.cc:2804] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:57.944254 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 4071ee02730f42cea7bafe35aa4ca63d (DEFAULT_TABLE table=default.Table_51cdbe440ce64783a1ead0a751db0e3e [id=3e8c9646322a4057a223b3fd184518b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:57.945395 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4071ee02730f42cea7bafe35aa4ca63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:57.905745 13114 raft_consensus.cc:493] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:57.948673 13114 raft_consensus.cc:515] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.954861 10862 tablet_service.cc:1511] Processing CreateTablet for tablet 4071ee02730f42cea7bafe35aa4ca63d (DEFAULT_TABLE table=default.Table_51cdbe440ce64783a1ead0a751db0e3e [id=3e8c9646322a4057a223b3fd184518b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:57.962268 13143 raft_consensus.cc:493] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:57.963038 13143 raft_consensus.cc:515] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:45:57.947925 13028 raft_consensus.cc:697] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:57.966557 13028 consensus_queue.cc:237] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:57.975844 11553 tablet_bootstrap.cc:492] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:57.976198 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c87361b29664464f9fd73be974e2d3a0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:57.979869 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a5ef5a3ff44f74ab19998ba9f19310" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
W20260809 01:45:57.981385 11063 leader_election.cc:343] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: a38bd344108649ed8e1687f82154000b
I20260809 01:45:57.982573 10597 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet cfb9afd3be194830ba99f5e20fdaca0a (table default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]). Expected version 1 got 0
I20260809 01:45:57.983789 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38bd344108649ed8e1687f82154000b" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:57.985073 11062 leader_election.cc:343] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:57.985441 11062 leader_election.cc:304] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:57.986967 12599 raft_consensus.cc:2749] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:57.953176 13142 tablet.cc:1724] T 85d0a9d73d4841c393e7d2cef41fee89 P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 01:45:57.990561 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c7ec3928224059ab7f58d203224536" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:57.990733 10855 tablet_service.cc:1511] Processing CreateTablet for tablet 655f526aa758420a8505368da62b3628 (DEFAULT_TABLE table=default.Table_d21ec0ae97254d549278ab5c7e3e6389 [id=80e638e9f782416389a651c2aa7c4c04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:57.996338 13114 leader_election.cc:290] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:57.983409 10597 catalog_manager.cc:5719] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66). New cstate: current_term: 1 leader_uuid: "33b5652538e64ed58a09993d3188f8d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:57.991873 13086 tablet_replica.cc:333] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:57.992381 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b2bb9c64aa1462eb4b14f0f11614149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.005347 13086 raft_consensus.cc:2243] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:58.006330 10862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4071ee02730f42cea7bafe35aa4ca63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.006800 13086 raft_consensus.cc:2272] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:58.007200 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 655f526aa758420a8505368da62b3628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.009182 13086 ts_tablet_manager.cc:1916] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:57.991746 10805 leader_election.cc:343] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:58.011606 13143 leader_election.cc:290] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:58.013742 11119 tablet_service.cc:1511] Processing CreateTablet for tablet 655f526aa758420a8505368da62b3628 (DEFAULT_TABLE table=default.Table_d21ec0ae97254d549278ab5c7e3e6389 [id=80e638e9f782416389a651c2aa7c4c04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.014469 10847 tablet_service.cc:1511] Processing CreateTablet for tablet 6180502b74b446ee89a9f3abf7ac9589 (DEFAULT_TABLE table=default.Table_25a94901452a4afba66db1eedb447239 [id=37dcb37f05bb4bb08c3df7d8505ac076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.014859 11119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 655f526aa758420a8505368da62b3628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.015501 10847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6180502b74b446ee89a9f3abf7ac9589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.016832 11107 tablet_service.cc:1511] Processing CreateTablet for tablet e1f606bb846f41608e598d5f45315a55 (DEFAULT_TABLE table=default.Table_5aed014baecb416686e9f81b1e185461 [id=045f4f6dbb8445ec8d7c2e5c52ea8d97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.017899 11107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1f606bb846f41608e598d5f45315a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.018239 11117 tablet_service.cc:1511] Processing CreateTablet for tablet 6180502b74b446ee89a9f3abf7ac9589 (DEFAULT_TABLE table=default.Table_25a94901452a4afba66db1eedb447239 [id=37dcb37f05bb4bb08c3df7d8505ac076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.019207 11117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6180502b74b446ee89a9f3abf7ac9589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.010336 10805 leader_election.cc:304] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:58.027688 10805 leader_election.cc:343] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:58.028388 10805 leader_election.cc:343] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:58.029452 13085 raft_consensus.cc:2749] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:58.028813 10805 leader_election.cc:304] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:58.030751 13085 raft_consensus.cc:2749] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:58.032284 11289 leader_election.cc:343] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: b9493892c1ff48e7a0558ff0fcb7a0f3
I20260809 01:45:58.037771 13085 raft_consensus.cc:493] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:58.040205 10867 tablet_service.cc:1511] Processing CreateTablet for tablet e1f606bb846f41608e598d5f45315a55 (DEFAULT_TABLE table=default.Table_5aed014baecb416686e9f81b1e185461 [id=045f4f6dbb8445ec8d7c2e5c52ea8d97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.041172 10867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1f606bb846f41608e598d5f45315a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.042078 13140 tablet.cc:1724] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 01:45:58.044378 10597 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet e3684fa56f9549f492a96a5c765eda78 (table default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]). Expected version 1 got 0
I20260809 01:45:58.044521 11554 tablet_bootstrap.cc:654] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:58.051038 11124 tablet_service.cc:1511] Processing CreateTablet for tablet afe1f037755943de8ef97e5268731c46 (DEFAULT_TABLE table=default.Table_6f67b7622b1640ec9bc317480a93535b [id=eab4b5fbb77849e88325a9ebd4d6e798]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.051985 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe1f037755943de8ef97e5268731c46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.053609 10591 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b
W20260809 01:45:58.053779 11470 tablet.cc:2406] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:45:58.055111 10591 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 425) took 1474 ms. Trace:
I20260809 01:45:58.058195 10592 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_b254da32a96842368d87fa797c32915a" } new_table_name: "default.Table_405329d656e44a29bbe12f4ce7752f18" modify_external_catalogs: true
I20260809 01:45:58.059762 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:56 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:58.062120 11105 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:56 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:58.055393 10591 rpcz_store.cc:276] 0809 01:45:56.580552 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:56.580701 (+ 149us) service_pool.cc:224] Handling call
0809 01:45:56.582008 (+ 1307us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:56.582726 (+ 718us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:56.582904 (+ 178us) catalog_manager.cc:2850] Locking tablets
0809 01:45:57.857567 (+1274663us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:58.053232 (+195665us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:58.053412 (+ 180us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:58.053501 (+ 89us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:58.055051 (+ 1550us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:57.877653 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:57.878024 (+ 371us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:57.878047 (+ 23us) write_op.cc:435] Acquired schema lock
0809 01:45:57.878060 (+ 13us) tablet.cc:662] Decoding operations
0809 01:45:57.878392 (+ 332us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:57.878430 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:57.878453 (+ 23us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:57.878898 (+ 445us) tablet.cc:701] Row locks acquired
0809 01:45:57.878912 (+ 14us) write_op.cc:260] PREPARE: finished
0809 01:45:57.879040 (+ 128us) write_op.cc:270] Start()
0809 01:45:57.879130 (+ 90us) write_op.cc:276] Timestamp: P: 1786239957879000 usec, L: 0
0809 01:45:57.879146 (+ 16us) op_driver.cc:348] REPLICATION: starting
0809 01:45:57.913631 (+ 34485us) log.cc:844] Serialized 756 byte log entry
0809 01:45:57.943868 (+ 30237us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678d951 00007fa2c6820208 00007fa2c6caff61 00007fa2c6caf738 00007fa2c6cb43d1 00007fa2c6cb4389 00007fa2c6cb4319 00007fa2c6cb42e1 00007fa2c6cb350d 00000000005bd8c4 00000000005bd6f9
0809 01:45:58.049937 (+106069us) op_driver.cc:464] REPLICATION: finished
0809 01:45:58.051173 (+ 1236us) write_op.cc:301] APPLY: starting
0809 01:45:58.051234 (+ 61us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:58.051339 (+ 105us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:58.051358 (+ 19us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:58.051648 (+ 290us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:58.051778 (+ 130us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:58.051818 (+ 40us) write_op.cc:312] APPLY: finished
0809 01:45:58.052010 (+ 192us) log.cc:844] Serialized 29 byte log entry
0809 01:45:58.052158 (+ 148us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:58.052222 (+ 64us) write_op.cc:454] Released schema lock
0809 01:45:58.052254 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:58.052464 (+ 210us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:58.052489 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1082,"apply.run_cpu_time_us":1266,"apply.run_wall_time_us":1266,"num_ops":2,"prepare.queue_time_us":18007,"prepare.run_cpu_time_us":4080,"prepare.run_wall_time_us":66413,"raft.queue_time_us":4439,"raft.run_cpu_time_us":666,"raft.run_wall_time_us":1734,"replication_time_us":170688,"spinlock_wait_cycles":76899584}]]}
I20260809 01:45:58.063045 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_1118918607d4448c8ec1a89b98973711"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:58.063702 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b already in progress: creating tablet
W20260809 01:45:58.064412 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1118918607d4448c8ec1a89b98973711 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:58.071458 10595 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet b6159d660d544b3e907a08372027c0e2
I20260809 01:45:58.072834 10595 catalog_manager.cc:6807] ReplaceTablet: tablet b6159d660d544b3e907a08372027c0e2 deleted and replaced by tablet 457ae94c3f9b4f4dbe8cd3101e4c578b
I20260809 01:45:58.074872 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet b6159d660d544b3e907a08372027c0e2 with delete_type TABLET_DATA_DELETED (replaced by tablet 457ae94c3f9b4f4dbe8cd3101e4c578b) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:58.078701 11105 tablet_service.cc:1558] Processing DeleteTablet for tablet b6159d660d544b3e907a08372027c0e2 with delete_type TABLET_DATA_DELETED (replaced by tablet 457ae94c3f9b4f4dbe8cd3101e4c578b) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:58.079334 12904 tablet_replica.cc:333] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:58.079857 10861 tablet_service.cc:1511] Processing CreateTablet for tablet afe1f037755943de8ef97e5268731c46 (DEFAULT_TABLE table=default.Table_6f67b7622b1640ec9bc317480a93535b [id=eab4b5fbb77849e88325a9ebd4d6e798]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.080042 12904 raft_consensus.cc:2243] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:58.080821 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe1f037755943de8ef97e5268731c46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.080878 12904 raft_consensus.cc:2272] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:58.083626 12904 ts_tablet_manager.cc:1916] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:58.085390 11554 tablet_bootstrap.cc:492] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:58.086063 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_75b1acafcd514d78982b343ff92e9861"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:58.087625 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_75b1acafcd514d78982b343ff92e9861 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:58.090013 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9493892c1ff48e7a0558ff0fcb7a0f3" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:58.090574 11155 raft_consensus.cc:2468] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:58.093888 11553 tablet_bootstrap.cc:654] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:58.103091 10856 tablet_service.cc:1511] Processing CreateTablet for tablet a9c3b06f67954ba68edcf505118eaaec (DEFAULT_TABLE table=default.Table_1b84c811beb04055bfcb7e4f37bb3702 [id=b328ad295fa1411b872de319d21eeb1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.105046 10856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9c3b06f67954ba68edcf505118eaaec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.105919 11554 ts_tablet_manager.cc:1403] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.191s user 0.006s sys 0.008s
W20260809 01:45:58.106946 11283 consensus_peers.cc:597] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7803a6aff02d42c1833f9b2cd7332b9f. This is attempt 1: this message will repeat every 5th retry.
W20260809 01:45:58.107719 11283 leader_election.cc:343] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: c528b45de0d14c19af5d6836b7a14f1c
I20260809 01:45:58.108042 11283 leader_election.cc:304] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:58.109186 11283 leader_election.cc:304] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:58.124106 11106 tablet_service.cc:1511] Processing CreateTablet for tablet 8bcd72c7dafa49ecbf4183775be51618 (DEFAULT_TABLE table=default.Table_0a2df154a003451eba03bf9545957996 [id=2c8612f637a04c14bd8520e5259f1508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.125136 11106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bcd72c7dafa49ecbf4183775be51618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.126963 10859 tablet_service.cc:1511] Processing CreateTablet for tablet 90c7b50d54ea49ba9313292c41e1bfd6 (DEFAULT_TABLE table=default.Table_840104257b8e416281e21b88d91837cb [id=0cf752da04854e2ab5d551ca60596939]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.128014 10859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90c7b50d54ea49ba9313292c41e1bfd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.130328 11553 tablet_bootstrap.cc:492] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:58.130338 11105 tablet_service.cc:1511] Processing CreateTablet for tablet a9c3b06f67954ba68edcf505118eaaec (DEFAULT_TABLE table=default.Table_1b84c811beb04055bfcb7e4f37bb3702 [id=b328ad295fa1411b872de319d21eeb1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.130715 11553 ts_tablet_manager.cc:1403] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.155s user 0.012s sys 0.003s
I20260809 01:45:58.131233 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9c3b06f67954ba68edcf505118eaaec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.132937 11553 raft_consensus.cc:359] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.138566 12904 ts_tablet_manager.cc:1929] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:45:58.139008 12904 log.cc:1199] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/b6159d660d544b3e907a08372027c0e2
I20260809 01:45:58.109939 13119 raft_consensus.cc:2749] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:58.140784 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 1b7c3dcab1594a87bd8d12e1dbfd8e3b (DEFAULT_TABLE table=default.Table_ddfd5630624142bc91b821055881a2ed [id=c120f8a5a7dd4061b8689fce4e204d76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.141752 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b7c3dcab1594a87bd8d12e1dbfd8e3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.141628 13114 raft_consensus.cc:2804] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:58.141983 13114 raft_consensus.cc:493] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:58.142390 13114 raft_consensus.cc:3060] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:58.081965 11149 raft_consensus.cc:2468] T 33a5ef5a3ff44f74ab19998ba9f19310 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 1.
I20260809 01:45:58.149575 11551 tablet_bootstrap.cc:492] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:58.150031 11551 ts_tablet_manager.cc:1403] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.313s user 0.021s sys 0.011s
I20260809 01:45:58.038412 13085 raft_consensus.cc:515] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.158846 13153 tablet.cc:1724] T 85d0a9d73d4841c393e7d2cef41fee89 P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 01:45:58.159792 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938bfd21b294fa6a25f0d8efc1ca29a" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:58.162925 11551 raft_consensus.cc:359] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.149394 13114 raft_consensus.cc:515] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.176222 13114 leader_election.cc:290] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:58.149111 10804 leader_election.cc:304] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:58.161105 10805 leader_election.cc:343] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:58.186460 13083 raft_consensus.cc:2804] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:58.134816 13086 ts_tablet_manager.cc:1929] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:58.189461 13086 log.cc:1199] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/c704d148da8d406aa824253a51b850c8
W20260809 01:45:58.190826 11289 leader_election.cc:343] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: b9493892c1ff48e7a0558ff0fcb7a0f3
I20260809 01:45:58.190838 13086 ts_tablet_manager.cc:1950] T c704d148da8d406aa824253a51b850c8 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:58.192569 10864 tablet_service.cc:1511] Processing CreateTablet for tablet 83dd3b3eb484407bbef0e54581b759dd (DEFAULT_TABLE table=default.Table_a1e1833d52934d0b90c8e6f5046d03df [id=59cbeb450bbc4fb99011118dbfe5809a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.193552 10864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83dd3b3eb484407bbef0e54581b759dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.196733 13086 tablet_replica.cc:333] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:58.197144 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet c704d148da8d406aa824253a51b850c8 (table default.Table_a21564a1b68c4bc9b78278dfa70ad1a9 [id=c064685015ef4b88ac9132f920977a57]) successfully deleted
I20260809 01:45:58.152878 13085 leader_election.cc:290] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:58.206100 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]) successfully deleted
W20260809 01:45:58.207098 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet f7d7a8f179244b3b9c9ca042af67e4c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f7d7a8f179244b3b9c9ca042af67e4c0 already in progress: creating tablet
I20260809 01:45:58.133502 11553 raft_consensus.cc:385] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:58.208915 11553 raft_consensus.cc:740] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:58.209863 11553 consensus_queue.cc:260] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.212244 11553 ts_tablet_manager.cc:1434] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.081s user 0.005s sys 0.002s
I20260809 01:45:58.222927 11554 raft_consensus.cc:359] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.207857 13086 raft_consensus.cc:2243] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:58.225131 13086 raft_consensus.cc:2272] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:58.173431 12904 ts_tablet_manager.cc:1950] T b6159d660d544b3e907a08372027c0e2 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:58.225880 11551 raft_consensus.cc:385] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:58.180279 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9493892c1ff48e7a0558ff0fcb7a0f3" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:45:58.226158 11551 raft_consensus.cc:740] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:58.227489 10870 tablet_service.cc:1511] Processing CreateTablet for tablet bdde640ea8c04e8381c71acae7ac809f (DEFAULT_TABLE table=default.Table_7dd2cf9954234c83b63d37aebee151d5 [id=1678462d93db45abb2817dcb1278732c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.231406 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdde640ea8c04e8381c71acae7ac809f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.228389 10862 tablet_service.cc:1511] Processing CreateTablet for tablet da8ae5e9f2b74e9f8ac49005812b1c36 (DEFAULT_TABLE table=default.Table_f4a23e78e1d54205b4e1d03c5dd16fbe [id=c8bc54017b8645d0933a8330257f37a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.232985 10862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da8ae5e9f2b74e9f8ac49005812b1c36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.170498 10858 tablet_service.cc:1511] Processing CreateTablet for tablet 8bcd72c7dafa49ecbf4183775be51618 (DEFAULT_TABLE table=default.Table_0a2df154a003451eba03bf9545957996 [id=2c8612f637a04c14bd8520e5259f1508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.235047 10864 tablet_service.cc:1511] Processing CreateTablet for tablet 37fdc6b7e6b545b49174c3ad96934224 (DEFAULT_TABLE table=default.Table_01a686662b59496fa2cbde4a0304f124 [id=530e31f43cd744dd86253addedb49b43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.235714 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bcd72c7dafa49ecbf4183775be51618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.242920 13085 raft_consensus.cc:493] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:58.243561 13085 raft_consensus.cc:515] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.236881 10864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37fdc6b7e6b545b49174c3ad96934224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.260757 11554 raft_consensus.cc:385] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:58.260962 11124 tablet_service.cc:1511] Processing CreateTablet for tablet a6313d1d9d8541abbc8c8d723ef75615 (DEFAULT_TABLE table=default.Table_1417e8aa601a422ea188f62cd2c3642f [id=bc0a4fa9d25d45c89f5717ec7fc168aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.264731 11554 raft_consensus.cc:740] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:58.269642 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6313d1d9d8541abbc8c8d723ef75615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.230477 11553 tablet_bootstrap.cc:492] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:58.273701 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 6ddb803901984b6e8b7e41a54ddd5fba (DEFAULT_TABLE table=default.Table_882931db60674255a2de681747e0a21e [id=dfa00b5804714312836df48f4db653db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.274796 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ddb803901984b6e8b7e41a54ddd5fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.250151 10871 tablet_service.cc:1511] Processing CreateTablet for tablet 712a7c26f2304697b6fba23767ede2a0 (DEFAULT_TABLE table=default.Table_f79203f002e14b909a945be94e3e4782 [id=613d8fd9bff64a8baa54ac9c611927dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.276170 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 712a7c26f2304697b6fba23767ede2a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.277415 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet b6159d660d544b3e907a08372027c0e2 (table default.Table_99bad24613e94b6f840f637bea4978fc [id=ddfe680283854b9484dc3e9d6e626ab9]) successfully deleted
I20260809 01:45:58.277599 11554 consensus_queue.cc:260] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.278086 11126 tablet_service.cc:1511] Processing CreateTablet for tablet 712a7c26f2304697b6fba23767ede2a0 (DEFAULT_TABLE table=default.Table_f79203f002e14b909a945be94e3e4782 [id=613d8fd9bff64a8baa54ac9c611927dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.279250 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 712a7c26f2304697b6fba23767ede2a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.282480 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100ad529186e4821b85ba28996933a8a" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:58.282727 11127 tablet_service.cc:1511] Processing CreateTablet for tablet da8ae5e9f2b74e9f8ac49005812b1c36 (DEFAULT_TABLE table=default.Table_f4a23e78e1d54205b4e1d03c5dd16fbe [id=c8bc54017b8645d0933a8330257f37a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.283717 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da8ae5e9f2b74e9f8ac49005812b1c36. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:58.284652 10804 leader_election.cc:343] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:58.296053 13114 raft_consensus.cc:493] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:58.297113 11554 ts_tablet_manager.cc:1434] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.191s user 0.006s sys 0.000s
I20260809 01:45:58.296591 13114 raft_consensus.cc:515] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.298061 11554 tablet_bootstrap.cc:492] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:58.213857 10847 tablet_service.cc:1511] Processing CreateTablet for tablet a6313d1d9d8541abbc8c8d723ef75615 (DEFAULT_TABLE table=default.Table_1417e8aa601a422ea188f62cd2c3642f [id=bc0a4fa9d25d45c89f5717ec7fc168aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.300076 11116 tablet_service.cc:1511] Processing CreateTablet for tablet 83dd3b3eb484407bbef0e54581b759dd (DEFAULT_TABLE table=default.Table_a1e1833d52934d0b90c8e6f5046d03df [id=59cbeb450bbc4fb99011118dbfe5809a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.301141 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83dd3b3eb484407bbef0e54581b759dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.309417 11120 tablet_service.cc:1511] Processing CreateTablet for tablet 23f97a412594405bb5dca58e58f5dbc2 (DEFAULT_TABLE table=default.Table_e09e4f1ce6624e9b91ade11edfcb3e80 [id=979c29485c7d474dbabe82d9ad1ce280]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.310469 11120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f97a412594405bb5dca58e58f5dbc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.311842 11105 tablet_service.cc:1511] Processing CreateTablet for tablet 37fdc6b7e6b545b49174c3ad96934224 (DEFAULT_TABLE table=default.Table_01a686662b59496fa2cbde4a0304f124 [id=530e31f43cd744dd86253addedb49b43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.313072 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37fdc6b7e6b545b49174c3ad96934224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.315476 10855 tablet_service.cc:1511] Processing CreateTablet for tablet 1b7c3dcab1594a87bd8d12e1dbfd8e3b (DEFAULT_TABLE table=default.Table_ddfd5630624142bc91b821055881a2ed [id=c120f8a5a7dd4061b8689fce4e204d76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.183557 11117 tablet_service.cc:1511] Processing CreateTablet for tablet 90c7b50d54ea49ba9313292c41e1bfd6 (DEFAULT_TABLE table=default.Table_840104257b8e416281e21b88d91837cb [id=0cf752da04854e2ab5d551ca60596939]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.318742 11117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90c7b50d54ea49ba9313292c41e1bfd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.321645 13083 raft_consensus.cc:697] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 LEADER]: Becoming Leader. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Running, Role: LEADER
I20260809 01:45:58.322312 11111 tablet_service.cc:1511] Processing CreateTablet for tablet 6ddb803901984b6e8b7e41a54ddd5fba (DEFAULT_TABLE table=default.Table_882931db60674255a2de681747e0a21e [id=dfa00b5804714312836df48f4db653db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.324162 11111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ddb803901984b6e8b7e41a54ddd5fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.285584 11155 raft_consensus.cc:3060] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:58.327878 13083 consensus_queue.cc:237] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.226828 11551 consensus_queue.cc:260] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.342043 10847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6313d1d9d8541abbc8c8d723ef75615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.326215 13085 leader_election.cc:290] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:58.310050 13086 ts_tablet_manager.cc:1916] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:58.324554 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 23f97a412594405bb5dca58e58f5dbc2 (DEFAULT_TABLE table=default.Table_e09e4f1ce6624e9b91ade11edfcb3e80 [id=979c29485c7d474dbabe82d9ad1ce280]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.349354 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet c528b45de0d14c19af5d6836b7a14f1c with delete_type TABLET_DATA_DELETED (replaced by tablet 31874b7e70a847a4885bd09da40a6902) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:58.350411 11341 tablet_service.cc:1511] Processing CreateTablet for tablet 0b2bb9c64aa1462eb4b14f0f11614149 (DEFAULT_TABLE table=default.Table_447d9518a2a7484ba63a2950dea459df [id=34c434cfe1034680b47af02d4ded4684]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:58.351948 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 48731.3us
W20260809 01:45:58.352900 11289 consensus_peers.cc:597] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7803a6aff02d42c1833f9b2cd7332b9f. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:58.354005 13139 raft_consensus.cc:493] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:58.354656 13139 raft_consensus.cc:515] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.356241 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100ad529186e4821b85ba28996933a8a" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:58.357640 10805 leader_election.cc:343] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:58.358232 10805 leader_election.cc:304] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:58.359221 13085 raft_consensus.cc:2749] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:58.349269 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 47056.9us
I20260809 01:45:58.363507 13114 leader_election.cc:290] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:58.365332 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f022e6a57a4ec0a5a8c33159e97a8e" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:58.372957 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f97a412594405bb5dca58e58f5dbc2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:58.374192 11289 leader_election.cc:343] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 403cb09c1786491ab0d40502038bf9c0
I20260809 01:45:58.375023 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b2bb9c64aa1462eb4b14f0f11614149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.373297 13139 leader_election.cc:290] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:58.377300 11349 tablet_service.cc:1558] Processing DeleteTablet for tablet 7803a6aff02d42c1833f9b2cd7332b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:37 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:58.378767 11349 tablet_service.cc:1558] Processing DeleteTablet for tablet b2726cb09f734ed18bb9379f94f3f22a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:41 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:58.380136 11349 tablet_service.cc:1558] Processing DeleteTablet for tablet c704d148da8d406aa824253a51b850c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:56 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:58.383136 11349 tablet_service.cc:1511] Processing CreateTablet for tablet 4071ee02730f42cea7bafe35aa4ca63d (DEFAULT_TABLE table=default.Table_51cdbe440ce64783a1ead0a751db0e3e [id=3e8c9646322a4057a223b3fd184518b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.375396 10903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f022e6a57a4ec0a5a8c33159e97a8e" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:58.386270 11349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4071ee02730f42cea7bafe35aa4ca63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.381282 10986 heartbeater.cc:344] Connected to a master server at 127.10.8.125:42063
I20260809 01:45:58.395799 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403cb09c1786491ab0d40502038bf9c0" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:58.396378 11156 raft_consensus.cc:2468] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:45:58.397841 11283 leader_election.cc:304] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:58.398936 11283 leader_election.cc:304] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:58.401798 13114 raft_consensus.cc:2749] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:58.402515 13114 raft_consensus.cc:2804] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:58.402892 13114 raft_consensus.cc:493] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:58.403280 13114 raft_consensus.cc:3060] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:58.409652 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938bfd21b294fa6a25f0d8efc1ca29a" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:58.409929 11336 tablet_service.cc:1511] Processing CreateTablet for tablet 655f526aa758420a8505368da62b3628 (DEFAULT_TABLE table=default.Table_d21ec0ae97254d549278ab5c7e3e6389 [id=80e638e9f782416389a651c2aa7c4c04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.411063 11336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 655f526aa758420a8505368da62b3628. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:58.421201 10580 catalog_manager.cc:5123] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): alter failed for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]),no further retry: Not found: Tablet not found: 7706838afc9f4dbebc3213b82c4da126
I20260809 01:45:58.423063 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 6180502b74b446ee89a9f3abf7ac9589 (DEFAULT_TABLE table=default.Table_25a94901452a4afba66db1eedb447239 [id=37dcb37f05bb4bb08c3df7d8505ac076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.423328 11346 tablet_service.cc:1511] Processing CreateTablet for tablet e1f606bb846f41608e598d5f45315a55 (DEFAULT_TABLE table=default.Table_5aed014baecb416686e9f81b1e185461 [id=045f4f6dbb8445ec8d7c2e5c52ea8d97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.424104 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6180502b74b446ee89a9f3abf7ac9589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.424696 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1f606bb846f41608e598d5f45315a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.426355 13028 raft_consensus.cc:493] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:58.427022 13028 raft_consensus.cc:515] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.431002 11551 ts_tablet_manager.cc:1434] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.281s user 0.005s sys 0.002s
I20260809 01:45:58.432197 11551 tablet_bootstrap.cc:492] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
W20260809 01:45:58.434192 10804 leader_election.cc:343] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:58.434578 10804 leader_election.cc:304] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:58.435799 13085 raft_consensus.cc:2749] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:58.436339 13085 raft_consensus.cc:493] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:58.436743 13085 raft_consensus.cc:515] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.438545 13085 leader_election.cc:290] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:58.440831 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b334f049e24768ad2291326ae70121" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:58.442071 10805 leader_election.cc:343] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:58.450148 11553 tablet_bootstrap.cc:654] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:58.453797 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b7c3dcab1594a87bd8d12e1dbfd8e3b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:58.454128 13153 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.274s user 0.001s sys 0.000s
I20260809 01:45:58.456362 11334 tablet_service.cc:1511] Processing CreateTablet for tablet afe1f037755943de8ef97e5268731c46 (DEFAULT_TABLE table=default.Table_6f67b7622b1640ec9bc317480a93535b [id=eab4b5fbb77849e88325a9ebd4d6e798]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.457497 11334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe1f037755943de8ef97e5268731c46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.462863 11345 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:56 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:58.464066 11345 tablet_service.cc:1558] Processing DeleteTablet for tablet b6159d660d544b3e907a08372027c0e2 with delete_type TABLET_DATA_DELETED (replaced by tablet 457ae94c3f9b4f4dbe8cd3101e4c578b) from {username='slave'} at 127.0.0.1:53476
W20260809 01:45:58.478467 11246 tablet.cc:2406] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:45:58.481822 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b334f049e24768ad2291326ae70121" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:58.488058 11155 raft_consensus.cc:2468] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:45:58.491171 13028 leader_election.cc:290] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:58.496156 10586 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 57) took 2148 ms (client timeout 1000 ms). Trace:
I20260809 01:45:58.496536 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet 91941616dfdd4df3aaa7c8460876d273 with delete_type TABLET_DATA_DELETED (replaced by tablet c11b0394533a485294ace9ee5e99bb19) from {username='slave'} at 127.0.0.1:56566
W20260809 01:45:58.496469 10586 rpcz_store.cc:269] 0809 01:45:56.348008 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:56.375161 (+ 27153us) service_pool.cc:224] Handling call
0809 01:45:56.550636 (+175475us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:58.496089 (+1945453us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":4,"spinlock_wait_cycles":398687104}
I20260809 01:45:58.506826 10903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63c1a610663a4a2fa58f6cdc5f8d86ce" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:58.507498 10903 raft_consensus.cc:2468] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
W20260809 01:45:58.514905 10804 leader_election.cc:343] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:58.515277 10804 leader_election.cc:304] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:58.517798 13083 raft_consensus.cc:2749] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:58.513618 11063 leader_election.cc:304] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:58.527997 12904 tablet_replica.cc:333] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:58.528980 12904 raft_consensus.cc:2243] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:58.529493 12904 raft_consensus.cc:2272] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:58.531697 12904 ts_tablet_manager.cc:1916] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:58.531983 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet b9493892c1ff48e7a0558ff0fcb7a0f3 with delete_type TABLET_DATA_DELETED (replaced by tablet 83dd3b3eb484407bbef0e54581b759dd) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:58.532861 10589 master_service.cc:960] ReplaceTablet: received request to replace tablet d18d1f8343ed4c33a70fccbecc3012a6 from {username='slave'} at 127.0.0.1:37956
W20260809 01:45:58.539299 10596 catalog_manager.cc:5911] Failed to send AlterTable RPC for tablet a04881d1c5254053bef50dd3d9d475f2 (table default.Table_57234dd818414b7397d1ac621bdb06a7 [id=17cbc8bf77e142d08d03bd7a051e00ad], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet a04881d1c5254053bef50dd3d9d475f2 consensus state has no leader
W20260809 01:45:58.540925 10596 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 58) took 1171 ms (client timeout 1000 ms). Trace:
I20260809 01:45:58.542356 11283 leader_election.cc:304] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:45:58.546236 10599 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.66:57751 (request call id 64) took 1772 ms (client timeout 1000 ms). Trace:
W20260809 01:45:58.546607 10599 rpcz_store.cc:269] 0809 01:45:56.721603 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:56.775120 (+ 53517us) service_pool.cc:224] Handling call
0809 01:45:56.801994 (+ 26874us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:58.493442 (+1691448us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15534,"reported_tablets":5,"spinlock_wait_cycles":44532480}
I20260809 01:45:58.551023 10870 tablet_service.cc:1558] Processing DeleteTablet for tablet 7706838afc9f4dbebc3213b82c4da126 with delete_type TABLET_DATA_DELETED (replaced by tablet cec52957d4f5460d9f0846bc976926db) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:58.551235 11111 tablet_service.cc:1558] Processing DeleteTablet for tablet 403cb09c1786491ab0d40502038bf9c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:43 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:58.554292 13028 consensus_queue.cc:1048] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 01:45:58.556185 10592 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1c5897b881c448dab0cafe9238fa4bbb consensus state has no leader
I20260809 01:45:58.562426 11119 tablet_service.cc:1511] Processing CreateTablet for tablet bdde640ea8c04e8381c71acae7ac809f (DEFAULT_TABLE table=default.Table_7dd2cf9954234c83b63d37aebee151d5 [id=1678462d93db45abb2817dcb1278732c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.550113 13143 raft_consensus.cc:2804] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:58.575152 11120 tablet_service.cc:1558] Processing DeleteTablet for tablet a38bd344108649ed8e1687f82154000b with delete_type TABLET_DATA_DELETED (replaced by tablet 878b7a9d92df4824902b2b15295f65df) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:58.576776 11119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdde640ea8c04e8381c71acae7ac809f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:58.577026 10597 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.66:57751 (request call id 65) took 838 ms (client timeout 1000 ms). Trace:
W20260809 01:45:58.541301 10596 rpcz_store.cc:269] 0809 01:45:57.369886 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:57.370033 (+ 147us) service_pool.cc:224] Handling call
0809 01:45:58.540845 (+1170812us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25822,"reported_tablets":11}
I20260809 01:45:58.575754 13143 raft_consensus.cc:493] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:58.582203 13143 raft_consensus.cc:3060] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:58.582057 10864 tablet_service.cc:1558] Processing DeleteTablet for tablet c704d148da8d406aa824253a51b850c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:56 UTC) from {username='slave'} at 127.0.0.1:45982
W20260809 01:45:58.588907 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 46749us
I20260809 01:45:58.590404 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63c1a610663a4a2fa58f6cdc5f8d86ce" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:58.592224 11062 leader_election.cc:343] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:58.595959 11062 consensus_peers.cc:597] T 2692a931500443d298e25a2b362b22a1 P 33b5652538e64ed58a09993d3188f8d7 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:58.601910 11395 raft_consensus.cc:3060] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:58.603492 11551 tablet_bootstrap.cc:654] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:58.605041 11111 tablet_service.cc:1558] Processing DeleteTablet for tablet c3bf43978dc94c2b9a7f3c18a4677c40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:43 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:58.609882 13161 tablet_replica.cc:333] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:58.610777 13161 raft_consensus.cc:2243] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:58.611312 13161 raft_consensus.cc:2272] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:58.613150 13161 ts_tablet_manager.cc:1916] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:58.629072 13143 raft_consensus.cc:515] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.594921 13139 raft_consensus.cc:2804] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:58.640918 11553 tablet_bootstrap.cc:492] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:58.651042 13139 raft_consensus.cc:697] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:45:58.652010 13139 consensus_queue.cc:237] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.641472 11553 ts_tablet_manager.cc:1403] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.411s user 0.005s sys 0.012s
I20260809 01:45:58.659066 11345 tablet_service.cc:1511] Processing CreateTablet for tablet a9c3b06f67954ba68edcf505118eaaec (DEFAULT_TABLE table=default.Table_1b84c811beb04055bfcb7e4f37bb3702 [id=b328ad295fa1411b872de319d21eeb1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.659444 11553 raft_consensus.cc:359] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.660019 11553 raft_consensus.cc:385] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:58.660247 11553 raft_consensus.cc:740] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:58.660964 11345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9c3b06f67954ba68edcf505118eaaec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.660844 11553 consensus_queue.cc:260] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:58.665091 11063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 133647us
I20260809 01:45:58.671181 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 90c7b50d54ea49ba9313292c41e1bfd6 (DEFAULT_TABLE table=default.Table_840104257b8e416281e21b88d91837cb [id=0cf752da04854e2ab5d551ca60596939]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.671532 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 1b7c3dcab1594a87bd8d12e1dbfd8e3b (DEFAULT_TABLE table=default.Table_ddfd5630624142bc91b821055881a2ed [id=c120f8a5a7dd4061b8689fce4e204d76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.672201 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90c7b50d54ea49ba9313292c41e1bfd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.672925 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b7c3dcab1594a87bd8d12e1dbfd8e3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.672803 11351 tablet_service.cc:1511] Processing CreateTablet for tablet a6313d1d9d8541abbc8c8d723ef75615 (DEFAULT_TABLE table=default.Table_1417e8aa601a422ea188f62cd2c3642f [id=bc0a4fa9d25d45c89f5717ec7fc168aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.674261 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6313d1d9d8541abbc8c8d723ef75615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.675221 11551 tablet_bootstrap.cc:492] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:58.675740 11551 ts_tablet_manager.cc:1403] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.244s user 0.007s sys 0.011s
I20260809 01:45:58.698010 13143 leader_election.cc:290] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:58.698279 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63c1a610663a4a2fa58f6cdc5f8d86ce" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:58.698503 10910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63c1a610663a4a2fa58f6cdc5f8d86ce" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:58.699105 10910 raft_consensus.cc:3060] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:58.701723 11062 leader_election.cc:343] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:58.706550 10910 raft_consensus.cc:2468] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:58.708117 11063 leader_election.cc:304] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:58.709944 13169 raft_consensus.cc:2804] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:58.710552 13169 raft_consensus.cc:697] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:58.711602 13169 consensus_queue.cc:237] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.722699 12904 ts_tablet_manager.cc:1929] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:58.723150 12904 log.cc:1199] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/91941616dfdd4df3aaa7c8460876d273
I20260809 01:45:58.723949 12904 ts_tablet_manager.cc:1950] T 91941616dfdd4df3aaa7c8460876d273 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:58.728998 13086 ts_tablet_manager.cc:1929] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:58.729770 13086 log.cc:1199] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/4ad9cdaa5c0a4dc7bfc34db8f18c2b6b
I20260809 01:45:58.730716 13086 ts_tablet_manager.cc:1950] T 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
W20260809 01:45:58.577580 10597 rpcz_store.cc:269] 0809 01:45:57.738675 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:57.763305 (+ 24630us) service_pool.cc:224] Handling call
0809 01:45:58.576924 (+813619us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:58.201623 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:58.201963 (+ 340us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:58.201987 (+ 24us) write_op.cc:435] Acquired schema lock
0809 01:45:58.201998 (+ 11us) tablet.cc:662] Decoding operations
0809 01:45:58.202283 (+ 285us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:58.202319 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:58.202340 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:58.202486 (+ 146us) tablet.cc:701] Row locks acquired
0809 01:45:58.202499 (+ 13us) write_op.cc:260] PREPARE: finished
0809 01:45:58.202602 (+ 103us) write_op.cc:270] Start()
0809 01:45:58.202682 (+ 80us) write_op.cc:276] Timestamp: P: 1786239958202528 usec, L: 0
0809 01:45:58.202697 (+ 15us) op_driver.cc:348] REPLICATION: starting
0809 01:45:58.203132 (+ 435us) log.cc:844] Serialized 527 byte log entry
0809 01:45:58.480896 (+277764us) op_driver.cc:464] REPLICATION: finished
0809 01:45:58.481745 (+ 849us) write_op.cc:301] APPLY: starting
0809 01:45:58.481813 (+ 68us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:58.481978 (+ 165us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:58.482079 (+ 101us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:58.482347 (+ 268us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:58.482513 (+ 166us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:58.482615 (+ 102us) write_op.cc:312] APPLY: finished
0809 01:45:58.482871 (+ 256us) log.cc:844] Serialized 23 byte log entry
0809 01:45:58.483113 (+ 242us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:58.483188 (+ 75us) write_op.cc:454] Released schema lock
0809 01:45:58.483240 (+ 52us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:58.483586 (+ 346us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:58.483634 (+ 48us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":654,"apply.run_cpu_time_us":1785,"apply.run_wall_time_us":1785,"num_ops":1,"prepare.queue_time_us":155284,"prepare.run_cpu_time_us":1682,"prepare.run_wall_time_us":2519,"replication_time_us":278110}]]}
I20260809 01:45:58.732188 13086 tablet_replica.cc:333] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:58.743099 11341 tablet_service.cc:1511] Processing CreateTablet for tablet 8bcd72c7dafa49ecbf4183775be51618 (DEFAULT_TABLE table=default.Table_0a2df154a003451eba03bf9545957996 [id=2c8612f637a04c14bd8520e5259f1508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.703897 11551 raft_consensus.cc:359] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.744251 11551 raft_consensus.cc:385] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:58.744575 11551 raft_consensus.cc:740] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:58.745416 11551 consensus_queue.cc:260] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.750064 13086 raft_consensus.cc:2243] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:58.752426 11551 ts_tablet_manager.cc:1434] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.076s user 0.005s sys 0.001s
I20260809 01:45:58.754123 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 91941616dfdd4df3aaa7c8460876d273 (table default.Table_ab740dbd782f44eb8f423baeede1de0f [id=92630f3e58394023a931a5e2c40e8591]) successfully deleted
I20260809 01:45:58.756708 11551 tablet_bootstrap.cc:492] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:58.757427 13086 raft_consensus.cc:2272] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:58.735101 11553 ts_tablet_manager.cc:1434] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.078s user 0.004s sys 0.000s
I20260809 01:45:58.759693 11553 tablet_bootstrap.cc:492] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:58.760031 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 83dd3b3eb484407bbef0e54581b759dd (DEFAULT_TABLE table=default.Table_a1e1833d52934d0b90c8e6f5046d03df [id=59cbeb450bbc4fb99011118dbfe5809a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.760593 11360 tablet_service.cc:1511] Processing CreateTablet for tablet da8ae5e9f2b74e9f8ac49005812b1c36 (DEFAULT_TABLE table=default.Table_f4a23e78e1d54205b4e1d03c5dd16fbe [id=c8bc54017b8645d0933a8330257f37a0]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:45:58.762480 11005 tablet.cc:2406] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:45:58.762385 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 37fdc6b7e6b545b49174c3ad96934224 (DEFAULT_TABLE table=default.Table_01a686662b59496fa2cbde4a0304f124 [id=530e31f43cd744dd86253addedb49b43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.762333 11353 tablet_service.cc:1511] Processing CreateTablet for tablet bdde640ea8c04e8381c71acae7ac809f (DEFAULT_TABLE table=default.Table_7dd2cf9954234c83b63d37aebee151d5 [id=1678462d93db45abb2817dcb1278732c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.768186 13086 ts_tablet_manager.cc:1916] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:58.755385 12904 tablet_replica.cc:333] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:58.785753 10586 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_8060ad8eb1be43f4a3c2e9dac705c0e3" } modify_external_catalogs: true
I20260809 01:45:58.786659 10586 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_8060ad8eb1be43f4a3c2e9dac705c0e3" } modify_external_catalogs: true
I20260809 01:45:58.788007 11553 tablet_bootstrap.cc:654] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:58.791553 11551 tablet_bootstrap.cc:654] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:58.791735 10591 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_ab740dbd782f44eb8f423baeede1de0f" } new_table_name: "default.Table_58a6cb5f8b8043519208cd2d2f222940" modify_external_catalogs: true
I20260809 01:45:58.796460 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_4673a33b16564a8a8bd22ab3e30c3eec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:58.797077 12904 raft_consensus.cc:2243] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:58.797607 12904 raft_consensus.cc:2272] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:58.798050 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4673a33b16564a8a8bd22ab3e30c3eec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:58.799273 12904 ts_tablet_manager.cc:1916] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:58.806326 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_a877cd3e3d1846b597c39578812fbdcf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:58.807837 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a877cd3e3d1846b597c39578812fbdcf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:58.818615 10599 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_b363ce92e78e4e45b8aa8562d738bdbd" } modify_external_catalogs: true
W20260809 01:45:58.820899 10580 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 1c5897b881c448dab0cafe9238fa4bbb (table default.Table_405329d656e44a29bbe12f4ce7752f18 [id=b098ff5e09a0421ca14522917ab6e4f1], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1c5897b881c448dab0cafe9238fa4bbb consensus state has no leader
I20260809 01:45:58.827527 10596 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_3c356f5b797347c1ab98aaca3d06f4eb" } new_table_name: "default.Table_33cbb047c61649c3943ed7add66fbe48" modify_external_catalogs: true
I20260809 01:45:58.851550 11553 tablet_bootstrap.cc:492] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:58.852011 11553 ts_tablet_manager.cc:1403] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.092s user 0.012s sys 0.000s
I20260809 01:45:58.857070 13153 tablet.cc:1724] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:58.860514 11553 raft_consensus.cc:359] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.861207 11553 raft_consensus.cc:385] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:58.868101 11551 tablet_bootstrap.cc:492] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:58.868587 11551 ts_tablet_manager.cc:1403] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.112s user 0.005s sys 0.006s
I20260809 01:45:58.874383 11551 raft_consensus.cc:359] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.875038 11551 raft_consensus.cc:385] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:58.875344 11551 raft_consensus.cc:740] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:58.875952 11551 consensus_queue.cc:260] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.861492 11553 raft_consensus.cc:740] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:45:58.889228 11553 consensus_queue.cc:260] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:58.895793 11553 ts_tablet_manager.cc:1434] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.043s user 0.004s sys 0.004s
I20260809 01:45:58.888407 12904 ts_tablet_manager.cc:1929] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:58.898181 12904 log.cc:1199] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/b9493892c1ff48e7a0558ff0fcb7a0f3
I20260809 01:45:58.897220 13188 raft_consensus.cc:493] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 629f002048f64491afdd57f72a466e59)
I20260809 01:45:58.900589 12904 ts_tablet_manager.cc:1950] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:58.906713 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet b9493892c1ff48e7a0558ff0fcb7a0f3 (table default.Table_a1e1833d52934d0b90c8e6f5046d03df [id=59cbeb450bbc4fb99011118dbfe5809a]) successfully deleted
I20260809 01:45:58.897851 11551 ts_tablet_manager.cc:1434] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.029s user 0.004s sys 0.004s
I20260809 01:45:58.908241 11551 tablet_bootstrap.cc:492] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:58.910944 13086 ts_tablet_manager.cc:1929] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:45:58.901434 13188 raft_consensus.cc:515] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:45:58.910828 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_2b0c8696f76f4997be51b001eaf80dc3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:58.908859 12904 tablet_replica.cc:333] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:58.920544 12904 raft_consensus.cc:2243] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:58.921108 12904 raft_consensus.cc:2272] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:58.924746 12904 ts_tablet_manager.cc:1916] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:58.930083 13086 log.cc:1199] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/b6159d660d544b3e907a08372027c0e2
I20260809 01:45:58.931074 13086 ts_tablet_manager.cc:1950] T b6159d660d544b3e907a08372027c0e2 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:45:58.938925 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 712a7c26f2304697b6fba23767ede2a0 (DEFAULT_TABLE table=default.Table_f79203f002e14b909a945be94e3e4782 [id=613d8fd9bff64a8baa54ac9c611927dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:58.962497 13188 leader_election.cc:290] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 71bc72deb1d2402cabd817dfe3b27e88 (127.10.8.125:42063), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:45:58.937325 11551 tablet_bootstrap.cc:654] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
W20260809 01:45:58.963282 11062 consensus_peers.cc:597] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:59.006822 11553 tablet_bootstrap.cc:492] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:45:59.017962 11347 tablet_service.cc:1511] Processing CreateTablet for tablet 6ddb803901984b6e8b7e41a54ddd5fba (DEFAULT_TABLE table=default.Table_882931db60674255a2de681747e0a21e [id=dfa00b5804714312836df48f4db653db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:59.018671 11337 tablet_service.cc:1511] Processing CreateTablet for tablet 23f97a412594405bb5dca58e58f5dbc2 (DEFAULT_TABLE table=default.Table_e09e4f1ce6624e9b91ade11edfcb3e80 [id=979c29485c7d474dbabe82d9ad1ce280]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:45:59.038626 12952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09a73ee8e8ec496a984efcab27715506" candidate_term: 3 candidate_status { last_received { term: 2 index: 496 } } ignore_live_leader: false dest_uuid: "71bc72deb1d2402cabd817dfe3b27e88" is_pre_election: true
I20260809 01:45:59.057690 11554 tablet_bootstrap.cc:700] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent opening tablet: real 0.317s user 0.004s sys 0.000s
I20260809 01:45:59.058687 11554 tablet_bootstrap.cc:654] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:59.062645 12904 ts_tablet_manager.cc:1929] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:59.063131 12904 log.cc:1199] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/403cb09c1786491ab0d40502038bf9c0
I20260809 01:45:59.066092 12904 ts_tablet_manager.cc:1950] T 403cb09c1786491ab0d40502038bf9c0 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:59.093118 13169 raft_consensus.cc:493] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:59.094262 13169 raft_consensus.cc:515] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:59.103953 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b0c8696f76f4997be51b001eaf80dc3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:58.745077 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bcd72c7dafa49ecbf4183775be51618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:59.140857 11337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f97a412594405bb5dca58e58f5dbc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:59.141422 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83dd3b3eb484407bbef0e54581b759dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:59.141970 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da8ae5e9f2b74e9f8ac49005812b1c36. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:59.142346 12956 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s user 0.102s sys 0.165s
W20260809 01:45:59.142684 12956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s user 0.103s sys 0.165s
I20260809 01:45:59.146052 11347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ddb803901984b6e8b7e41a54ddd5fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:59.149780 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdde640ea8c04e8381c71acae7ac809f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:59.153793 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37fdc6b7e6b545b49174c3ad96934224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:45:58.749832 13114 raft_consensus.cc:515] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.099071 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fffa48ccf58476da79dccd9c008b67c" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:59.175679 11062 leader_election.cc:343] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:59.129462 11369 tablet_service.cc:1558] Processing DeleteTablet for tablet b9493892c1ff48e7a0558ff0fcb7a0f3 with delete_type TABLET_DATA_DELETED (replaced by tablet 83dd3b3eb484407bbef0e54581b759dd) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:59.193131 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 712a7c26f2304697b6fba23767ede2a0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:45:59.218888 11334 consensus_meta.cc:287] T afe1f037755943de8ef97e5268731c46 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.749s user 0.006s sys 0.000s
W20260809 01:45:59.221194 13191 log.cc:927] Time spent T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7: Append to log took a long time: real 0.196s user 0.001s sys 0.000s
I20260809 01:45:59.247761 10595 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:37956 (request call id 444) took 1273 ms. Trace:
I20260809 01:45:59.248242 10595 rpcz_store.cc:276] 0809 01:45:57.974841 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.085693 (+110852us) service_pool.cc:224] Handling call
0809 01:45:58.088037 (+ 2344us) catalog_manager.cc:2127] Acquired catalog manager lock
0809 01:45:58.089422 (+ 1385us) catalog_manager.cc:2173] Created new table and tablet info
0809 01:45:58.946560 (+857138us) catalog_manager.cc:2233] Wrote table and tablets to system table
0809 01:45:59.247436 (+300876us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0809 01:45:59.247699 (+ 263us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:58.587045 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:58.587408 (+ 363us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:58.587430 (+ 22us) write_op.cc:435] Acquired schema lock
0809 01:45:58.587442 (+ 12us) tablet.cc:662] Decoding operations
0809 01:45:58.588075 (+ 633us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:58.588114 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:58.588234 (+ 120us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:58.588425 (+ 191us) tablet.cc:701] Row locks acquired
0809 01:45:58.588438 (+ 13us) write_op.cc:260] PREPARE: finished
0809 01:45:58.589544 (+ 1106us) write_op.cc:270] Start()
0809 01:45:58.589663 (+ 119us) write_op.cc:276] Timestamp: P: 1786239958588503 usec, L: 0
0809 01:45:58.589690 (+ 27us) op_driver.cc:348] REPLICATION: starting
0809 01:45:58.840108 (+250418us) log.cc:844] Serialized 433 byte log entry
0809 01:45:58.938213 (+ 98105us) op_driver.cc:464] REPLICATION: finished
0809 01:45:58.938580 (+ 367us) write_op.cc:301] APPLY: starting
0809 01:45:58.938633 (+ 53us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:58.938729 (+ 96us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:58.938833 (+ 104us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:58.939269 (+ 436us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:58.939419 (+ 150us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 01:45:58.939456 (+ 37us) write_op.cc:312] APPLY: finished
0809 01:45:58.939634 (+ 178us) log.cc:844] Serialized 29 byte log entry
0809 01:45:58.939778 (+ 144us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:58.939851 (+ 73us) write_op.cc:454] Released schema lock
0809 01:45:58.939887 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:59.080973 (+141086us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678d951 00007fa2c6820208 00007fa2c6caff61 00007fa2c6caf738 00007fa2c6cb43d1 00007fa2c6cb4389 00007fa2c6cb4319 00007fa2c6cb42e1 00007fa2c6cb350d 00000000005bd8c4 00000000005bd6f9
0809 01:45:59.081745 (+ 772us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:59.081792 (+ 47us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":210,"apply.run_cpu_time_us":1630,"apply.run_wall_time_us":9045,"num_ops":2,"prepare.queue_time_us":437108,"prepare.run_cpu_time_us":5361,"prepare.run_wall_time_us":494685,"raft.queue_time_us":107,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":294,"replication_time_us":348408,"spinlock_wait_cycles":573463296}]]}
I20260809 01:45:59.232051 11551 tablet_bootstrap.cc:492] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:59.266943 11551 ts_tablet_manager.cc:1403] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.359s user 0.012s sys 0.004s
I20260809 01:45:59.270092 11551 raft_consensus.cc:359] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.270777 11551 raft_consensus.cc:385] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:59.271071 11551 raft_consensus.cc:740] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:59.272621 13169 leader_election.cc:290] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:59.111235 11346 consensus_meta.cc:287] T e1f606bb846f41608e598d5f45315a55 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.658s user 0.008s sys 0.000s
W20260809 01:45:59.286285 11351 consensus_meta.cc:287] T a6313d1d9d8541abbc8c8d723ef75615 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.560s user 0.008s sys 0.000s
I20260809 01:45:59.298707 12904 tablet_replica.cc:333] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:59.299513 12904 raft_consensus.cc:2243] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:59.299947 12904 raft_consensus.cc:2272] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:59.271813 11551 consensus_queue.cc:260] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.306427 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 403cb09c1786491ab0d40502038bf9c0 (table default.Table_63e5e407241d4eaea93f27d8285ab33a [id=9e066a2a246442b78bfc003520f4cfbd]) successfully deleted
I20260809 01:45:59.306550 13189 tablet.cc:1724] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:59.287026 13139 raft_consensus.cc:493] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:59.308064 13139 raft_consensus.cc:515] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.256171 10599 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_b363ce92e78e4e45b8aa8562d738bdbd" } modify_external_catalogs: true
I20260809 01:45:59.257038 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09a73ee8e8ec496a984efcab27715506" candidate_term: 3 candidate_status { last_received { term: 2 index: 496 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:45:59.281412 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b (table default.Table_0bcdcc37d5f04f40ab4dd02b66baf4a6 [id=97ccdb35be6a437288222feb89f9e751]) successfully deleted
I20260809 01:45:59.310287 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet b6159d660d544b3e907a08372027c0e2 (table default.Table_99bad24613e94b6f840f637bea4978fc [id=ddfe680283854b9484dc3e9d6e626ab9]) successfully deleted
W20260809 01:45:59.310997 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 7706838afc9f4dbebc3213b82c4da126 because the tablet was not found. No further retry: Not found: Tablet not found: 7706838afc9f4dbebc3213b82c4da126
W20260809 01:45:59.312240 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet c704d148da8d406aa824253a51b850c8 because the tablet was not found. No further retry: Not found: Tablet not found: c704d148da8d406aa824253a51b850c8
I20260809 01:45:59.313452 10319 leader_election.cc:304] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09a73ee8e8ec496a984efcab27715506; no voters: 629f002048f64491afdd57f72a466e59, 71bc72deb1d2402cabd817dfe3b27e88
I20260809 01:45:59.314474 13188 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 09a73ee8e8ec496a984efcab27715506 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 01:45:59.314779 11551 ts_tablet_manager.cc:1434] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.047s user 0.005s sys 0.000s
I20260809 01:45:59.315152 12904 ts_tablet_manager.cc:1916] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:59.315690 11551 tablet_bootstrap.cc:492] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:59.324746 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bc7dc3dfc2f4c1286ca24b8e3b20719" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:59.334236 11283 leader_election.cc:343] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:59.338945 10598 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:37956 (request call id 445) took 1228 ms. Trace:
I20260809 01:45:59.344568 10595 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet e3684fa56f9549f492a96a5c765eda78 (table default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]). Expected version 1 got 0
I20260809 01:45:59.339269 10598 rpcz_store.cc:276] 0809 01:45:58.110219 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.341375 (+231156us) service_pool.cc:224] Handling call
0809 01:45:58.796361 (+454986us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c86951c6 00007fa2c6249034 00007fa2c6248fc2 00007fa2c6248ef1 00007fa2c6248e6c 00007fa2c6248122 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:58.798453 (+ 2092us) catalog_manager.cc:2127] Acquired catalog manager lock
0809 01:45:58.800777 (+ 2324us) catalog_manager.cc:2173] Created new table and tablet info
0809 01:45:59.337199 (+536422us) catalog_manager.cc:2233] Wrote table and tablets to system table
0809 01:45:59.337487 (+ 288us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0809 01:45:59.338888 (+ 1401us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:59.082134 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:59.082693 (+ 559us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:59.082726 (+ 33us) write_op.cc:435] Acquired schema lock
0809 01:45:59.082745 (+ 19us) tablet.cc:662] Decoding operations
0809 01:45:59.083106 (+ 361us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:59.083162 (+ 56us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:59.083197 (+ 35us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:59.083460 (+ 263us) tablet.cc:701] Row locks acquired
0809 01:45:59.083491 (+ 31us) write_op.cc:260] PREPARE: finished
0809 01:45:59.083616 (+ 125us) write_op.cc:270] Start()
0809 01:45:59.083717 (+ 101us) write_op.cc:276] Timestamp: P: 1786239959083552 usec, L: 0
0809 01:45:59.083737 (+ 20us) op_driver.cc:348] REPLICATION: starting
0809 01:45:59.257921 (+174184us) log.cc:844] Serialized 433 byte log entry
0809 01:45:59.334446 (+ 76525us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b18000bb8a0. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c85b3c01 00007fa2c85c0581 00007fa2c6cb0dfd 00007fa2c6cb30fa 00007fa2c6cb3088 00007fa2c6cb3008 00007fa2c6cb2fc3 00007fa2c6cb21dc 00007fa2c8642a16 00007fa2c863d1c8 00007fa2c68306f0
0809 01:45:59.334487 (+ 41us) op_driver.cc:464] REPLICATION: finished
0809 01:45:59.335129 (+ 642us) write_op.cc:301] APPLY: starting
0809 01:45:59.335195 (+ 66us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:59.335312 (+ 117us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:59.335336 (+ 24us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:59.335781 (+ 445us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:59.335916 (+ 135us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 01:45:59.335956 (+ 40us) write_op.cc:312] APPLY: finished
0809 01:45:59.336157 (+ 201us) log.cc:844] Serialized 29 byte log entry
0809 01:45:59.336300 (+ 143us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:59.336377 (+ 77us) write_op.cc:454] Released schema lock
0809 01:45:59.336426 (+ 49us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":1044226432,"child_traces":[["op",{"apply.queue_time_us":477,"num_ops":2,"prepare.queue_time_us":262029,"prepare.run_cpu_time_us":2508,"prepare.run_wall_time_us":178609,"raft.queue_time_us":83,"raft.run_cpu_time_us":593,"raft.run_wall_time_us":593,"replication_time_us":182337,"spinlock_wait_cycles":24225536}]]}
I20260809 01:45:59.349807 13114 leader_election.cc:290] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:59.351186 13139 leader_election.cc:290] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:59.357136 11283 leader_election.cc:343] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: 403cb09c1786491ab0d40502038bf9c0
I20260809 01:45:59.345247 10595 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet cfb9afd3be194830ba99f5e20fdaca0a (table default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]). Expected version 1 got 0
W20260809 01:45:59.372469 11063 leader_election.cc:343] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 9fffa48ccf58476da79dccd9c008b67c
I20260809 01:45:59.372872 11063 leader_election.cc:304] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:59.374431 13143 raft_consensus.cc:2749] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:59.374967 13143 consensus_queue.cc:1048] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:59.379684 10598 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet e3684fa56f9549f492a96a5c765eda78 (table default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]). Expected version 1 got 0
I20260809 01:45:59.380470 10598 catalog_manager.cc:5719] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66). New cstate: current_term: 1 leader_uuid: "33b5652538e64ed58a09993d3188f8d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } }
I20260809 01:45:59.384994 12904 ts_tablet_manager.cc:1929] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:59.402221 12904 log.cc:1199] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/a38bd344108649ed8e1687f82154000b
I20260809 01:45:59.403652 12904 ts_tablet_manager.cc:1950] T a38bd344108649ed8e1687f82154000b P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:59.411587 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet a38bd344108649ed8e1687f82154000b (table default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]) successfully deleted
I20260809 01:45:59.416257 12904 tablet_replica.cc:333] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:45:59.417742 12904 raft_consensus.cc:2243] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:45:59.425808 12904 raft_consensus.cc:2272] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:59.428922 10586 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet f4985b1715f64adaae432d46dee4ea38
I20260809 01:45:59.430507 10586 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 446) took 1301 ms. Trace:
I20260809 01:45:59.431699 12904 ts_tablet_manager.cc:1916] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:59.440613 11119 tablet_service.cc:1558] Processing DeleteTablet for tablet f4985b1715f64adaae432d46dee4ea38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:58 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:59.431308 10586 rpcz_store.cc:276] 0809 01:45:58.129040 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.497339 (+368299us) service_pool.cc:224] Handling call
0809 01:45:58.785641 (+288302us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c86954d6 00007fa2c624db34 00007fa2c624dac2 00007fa2c624d9f1 00007fa2c624d96c 00007fa2c624cc22 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:58.787305 (+ 1664us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:58.788680 (+ 1375us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:58.788859 (+ 179us) catalog_manager.cc:2850] Locking tablets
0809 01:45:58.835118 (+ 46259us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:59.428551 (+593433us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:59.428738 (+ 187us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:59.428832 (+ 94us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:59.430451 (+ 1619us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:59.276904 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:59.277288 (+ 384us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:59.277312 (+ 24us) write_op.cc:435] Acquired schema lock
0809 01:45:59.277325 (+ 13us) tablet.cc:662] Decoding operations
0809 01:45:59.278226 (+ 901us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:59.278263 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:59.278284 (+ 21us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:59.278584 (+ 300us) tablet.cc:701] Row locks acquired
0809 01:45:59.278598 (+ 14us) write_op.cc:260] PREPARE: finished
0809 01:45:59.278702 (+ 104us) write_op.cc:270] Start()
0809 01:45:59.278776 (+ 74us) write_op.cc:276] Timestamp: P: 1786239959278628 usec, L: 0
0809 01:45:59.278791 (+ 15us) op_driver.cc:348] REPLICATION: starting
0809 01:45:59.391448 (+112657us) log.cc:844] Serialized 756 byte log entry
0809 01:45:59.392375 (+ 927us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678d951 00007fa2c6820208 00007fa2c6caff61 00007fa2c6caf738 00007fa2c6cb43d1 00007fa2c6cb4389 00007fa2c6cb4319 00007fa2c6cb42e1 00007fa2c6cb350d 00000000005bd8c4 00000000005bd6f9
0809 01:45:59.402111 (+ 9736us) op_driver.cc:464] REPLICATION: finished
0809 01:45:59.413487 (+ 11376us) write_op.cc:301] APPLY: starting
0809 01:45:59.413563 (+ 76us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:59.413672 (+ 109us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:59.413692 (+ 20us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:59.413957 (+ 265us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:59.414129 (+ 172us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:59.414173 (+ 44us) write_op.cc:312] APPLY: finished
0809 01:45:59.414339 (+ 166us) log.cc:844] Serialized 29 byte log entry
0809 01:45:59.414485 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:59.428286 (+ 13801us) write_op.cc:454] Released schema lock
0809 01:45:59.428342 (+ 56us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":660136832,"child_traces":[["op",{"apply.queue_time_us":11211,"apply.run_cpu_time_us":1371,"apply.run_wall_time_us":27361,"num_ops":2,"prepare.queue_time_us":439697,"prepare.run_cpu_time_us":3203,"prepare.run_wall_time_us":115647,"raft.queue_time_us":100,"raft.run_cpu_time_us":634,"raft.run_wall_time_us":634,"replication_time_us":123234,"spinlock_wait_cycles":257851648}]]}
I20260809 01:45:59.447875 10598 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 63c1a610663a4a2fa58f6cdc5f8d86ce (table default.Table_bd6eedffdf6440efa3e776b7079015d8 [id=8a2084683a7740db80e2c52d3f4c9506]). Expected version 1 got 0
W20260809 01:45:59.449103 11345 consensus_meta.cc:287] T a9c3b06f67954ba68edcf505118eaaec P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.668s user 0.000s sys 0.010s
I20260809 01:45:59.453043 11551 tablet_bootstrap.cc:654] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:59.451211 11349 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 365) took 1070 ms. Trace:
I20260809 01:45:59.453476 11349 rpcz_store.cc:276] 0809 01:45:58.381005 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.381116 (+ 111us) service_pool.cc:224] Handling call
0809 01:45:58.385795 (+ 4679us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:58.385898 (+ 103us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:59.197678 (+811780us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:59.450702 (+253024us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:59.451137 (+ 435us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:59.451561 10598 catalog_manager.cc:5719] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66). New cstate: current_term: 1 leader_uuid: "33b5652538e64ed58a09993d3188f8d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } }
W20260809 01:45:59.458233 11336 consensus_meta.cc:287] T 655f526aa758420a8505368da62b3628 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.772s user 0.010s sys 0.000s
I20260809 01:45:59.459107 11336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53476 (request call id 366) took 1050 ms. Trace:
I20260809 01:45:59.459431 11336 rpcz_store.cc:276] 0809 01:45:58.409014 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.409130 (+ 116us) service_pool.cc:224] Handling call
0809 01:45:58.410613 (+ 1483us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:58.410708 (+ 95us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:58.685857 (+275149us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:59.458677 (+772820us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:59.459053 (+ 376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:59.459434 11554 tablet_bootstrap.cc:492] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:59.460486 11554 ts_tablet_manager.cc:1403] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 1.163s user 0.022s sys 0.011s
W20260809 01:45:59.459774 10591 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c11b0394533a485294ace9ee5e99bb19 consensus state has no leader
I20260809 01:45:59.471943 13209 tablet.cc:1724] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:45:59.474444 13168 raft_consensus.cc:493] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:59.474889 13168 raft_consensus.cc:515] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.477857 13168 leader_election.cc:290] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:59.480363 13208 raft_consensus.cc:493] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:59.492417 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet d18d1f8343ed4c33a70fccbecc3012a6
I20260809 01:45:59.493848 10589 catalog_manager.cc:6807] ReplaceTablet: tablet d18d1f8343ed4c33a70fccbecc3012a6 deleted and replaced by tablet d9025e61632c461c87972f879db46c87
I20260809 01:45:59.495312 10589 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:37956 (request call id 447) took 1312 ms. Trace:
I20260809 01:45:59.495579 10589 rpcz_store.cc:276] 0809 01:45:58.182940 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.532639 (+349699us) service_pool.cc:224] Handling call
0809 01:45:58.815835 (+283196us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8697d41 00007fa2c6269d37 00007fa2c6269cc2 00007fa2c6269bf1 00007fa2c6269b6c 00007fa2c6268e22 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:59.495265 (+679430us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:59.469908 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:59.470366 (+ 458us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:59.470393 (+ 27us) write_op.cc:435] Acquired schema lock
0809 01:45:59.470408 (+ 15us) tablet.cc:662] Decoding operations
0809 01:45:59.475832 (+ 5424us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:59.475872 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:59.475905 (+ 33us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:59.477217 (+ 1312us) tablet.cc:701] Row locks acquired
0809 01:45:59.477233 (+ 16us) write_op.cc:260] PREPARE: finished
0809 01:45:59.477449 (+ 216us) write_op.cc:270] Start()
0809 01:45:59.477549 (+ 100us) write_op.cc:276] Timestamp: P: 1786239959477288 usec, L: 0
0809 01:45:59.477564 (+ 15us) op_driver.cc:348] REPLICATION: starting
0809 01:45:59.478992 (+ 1428us) log.cc:844] Serialized 622 byte log entry
0809 01:45:59.489996 (+ 11004us) op_driver.cc:464] REPLICATION: finished
0809 01:45:59.490214 (+ 218us) write_op.cc:301] APPLY: starting
0809 01:45:59.490270 (+ 56us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:59.490374 (+ 104us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:59.490468 (+ 94us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:59.490810 (+ 342us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:59.491003 (+ 193us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:59.491045 (+ 42us) write_op.cc:312] APPLY: finished
0809 01:45:59.491221 (+ 176us) log.cc:844] Serialized 29 byte log entry
0809 01:45:59.491356 (+ 135us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:59.491430 (+ 74us) write_op.cc:454] Released schema lock
0809 01:45:59.491463 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:59.491670 (+ 207us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:59.491699 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":647940352,"child_traces":[["op",{"apply.queue_time_us":75,"apply.run_cpu_time_us":1423,"apply.run_wall_time_us":1423,"num_ops":2,"prepare.queue_time_us":71347,"prepare.run_cpu_time_us":2311,"prepare.run_wall_time_us":90030,"raft.queue_time_us":3267,"raft.run_cpu_time_us":647,"raft.run_wall_time_us":648,"replication_time_us":12338}]]}
I20260809 01:45:59.499233 11119 tablet_service.cc:1558] Processing DeleteTablet for tablet d18d1f8343ed4c33a70fccbecc3012a6 with delete_type TABLET_DATA_DELETED (replaced by tablet d9025e61632c461c87972f879db46c87) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:59.500003 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761370ec8b164fd184a2e091030f610b" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:59.501250 11062 leader_election.cc:343] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:59.502478 11351 tablet_service.cc:1558] Processing DeleteTablet for tablet f4985b1715f64adaae432d46dee4ea38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:58 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:59.503203 11554 raft_consensus.cc:359] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.503808 11554 raft_consensus.cc:385] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 01:45:59.505002 11283 consensus_peers.cc:597] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b9493892c1ff48e7a0558ff0fcb7a0f3. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:59.516798 11351 tablet_service.cc:1558] Processing DeleteTablet for tablet d18d1f8343ed4c33a70fccbecc3012a6 with delete_type TABLET_DATA_DELETED (replaced by tablet d9025e61632c461c87972f879db46c87) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:59.518404 11551 tablet_bootstrap.cc:492] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:59.521469 13161 ts_tablet_manager.cc:1929] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:59.525403 13161 log.cc:1199] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/c528b45de0d14c19af5d6836b7a14f1c
I20260809 01:45:59.526405 13161 ts_tablet_manager.cc:1950] T c528b45de0d14c19af5d6836b7a14f1c P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:45:59.526875 11465 heartbeater.cc:344] Connected to a master server at 127.10.8.125:42063
I20260809 01:45:59.528725 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 361) took 1223 ms. Trace:
I20260809 01:45:59.505013 11554 raft_consensus.cc:740] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:59.530014 11554 consensus_queue.cc:260] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.530575 11551 ts_tablet_manager.cc:1403] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.215s user 0.007s sys 0.003s
W20260809 01:45:59.537930 10596 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e3690212d43543fc982815dcff225a51 consensus state has no leader
I20260809 01:45:59.541836 10589 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_2142392a3c3b4314863176189fc985d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:59.543437 10589 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2142392a3c3b4314863176189fc985d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:59.555490 11554 ts_tablet_manager.cc:1434] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.093s user 0.005s sys 0.000s
I20260809 01:45:59.556660 11554 tablet_bootstrap.cc:492] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:59.557740 11551 raft_consensus.cc:359] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.560940 13206 raft_consensus.cc:493] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:59.561601 13206 raft_consensus.cc:515] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.566124 10847 tablet_service.cc:1558] Processing DeleteTablet for tablet f4985b1715f64adaae432d46dee4ea38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:58 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:59.570147 11551 raft_consensus.cc:385] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:59.570484 11551 raft_consensus.cc:740] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:59.572602 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet d18d1f8343ed4c33a70fccbecc3012a6 with delete_type TABLET_DATA_DELETED (replaced by tablet d9025e61632c461c87972f879db46c87) from {username='slave'} at 127.0.0.1:45982
I20260809 01:45:59.573220 13083 raft_consensus.cc:493] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:59.573710 13083 raft_consensus.cc:515] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.575760 13083 leader_election.cc:290] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:59.583300 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60176118d61548ff96b001ab0dae7d21" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:59.571167 11551 consensus_queue.cc:260] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.595518 11551 ts_tablet_manager.cc:1434] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.062s user 0.002s sys 0.002s
I20260809 01:45:59.598204 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_ba10600fdca746b3a5a091b2c78588a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:59.574743 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b02bd8c93b43bf90ff0192a9541aae" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:45:59.599725 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba10600fdca746b3a5a091b2c78588a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:45:59.599782 10804 leader_election.cc:343] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:59.600382 10804 leader_election.cc:343] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:59.609647 13208 raft_consensus.cc:515] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.612082 13208 leader_election.cc:290] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:59.618530 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa12353c25cb4da680d478ce104bc6ba" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:59.626545 10599 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 8e51d670bb6b4198aa37240b4843e6de
I20260809 01:45:59.626884 10599 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 448) took 1358 ms. Trace:
I20260809 01:45:59.627156 10599 rpcz_store.cc:276] 0809 01:45:58.268334 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.547974 (+279640us) service_pool.cc:224] Handling call
0809 01:45:58.818490 (+270516us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c86954d6 00007fa2c624db34 00007fa2c624dac2 00007fa2c624d9f1 00007fa2c624d96c 00007fa2c624cc22 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:45:59.309109 (+490619us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:59.382220 (+ 73111us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:59.382382 (+ 162us) catalog_manager.cc:2850] Locking tablets
0809 01:45:59.382914 (+ 532us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:59.398664 (+ 15750us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b5400030300. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c6be3b41 00007fa2c6cb8569 00007fa2c6cae18d 00007fa2c6c6f7eb 00007fa2c6c6f5c4 00007fa2c86ccb51 00007fa2c86cd3af 00007fa2c8576fd2 00007fa2c8575dbb 00007fa2c857398e 00007fa2c85733e3
0809 01:45:59.626214 (+227550us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:59.626386 (+ 172us) catalog_manager.cc:2898] Committing in-memory state
0809 01:45:59.626469 (+ 83us) catalog_manager.cc:2902] Aborting table tasks
0809 01:45:59.626830 (+ 361us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:59.575186 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:59.575550 (+ 364us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:59.575574 (+ 24us) write_op.cc:435] Acquired schema lock
0809 01:45:59.575590 (+ 16us) tablet.cc:662] Decoding operations
0809 01:45:59.575883 (+ 293us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:59.575930 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:59.575952 (+ 22us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:59.576141 (+ 189us) tablet.cc:701] Row locks acquired
0809 01:45:59.576156 (+ 15us) write_op.cc:260] PREPARE: finished
0809 01:45:59.576255 (+ 99us) write_op.cc:270] Start()
0809 01:45:59.576394 (+ 139us) write_op.cc:276] Timestamp: P: 1786239959576196 usec, L: 0
0809 01:45:59.576410 (+ 16us) op_driver.cc:348] REPLICATION: starting
0809 01:45:59.597393 (+ 20983us) log.cc:844] Serialized 533 byte log entry
0809 01:45:59.624466 (+ 27073us) op_driver.cc:464] REPLICATION: finished
0809 01:45:59.624816 (+ 350us) write_op.cc:301] APPLY: starting
0809 01:45:59.624876 (+ 60us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:59.624990 (+ 114us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:59.625095 (+ 105us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:59.625338 (+ 243us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:59.625489 (+ 151us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:59.625564 (+ 75us) write_op.cc:312] APPLY: finished
0809 01:45:59.625759 (+ 195us) log.cc:844] Serialized 29 byte log entry
0809 01:45:59.625910 (+ 151us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:59.625981 (+ 71us) write_op.cc:454] Released schema lock
0809 01:45:59.626017 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:59.626269 (+ 252us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:59.626292 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":651576064,"child_traces":[["op",{"apply.queue_time_us":200,"apply.run_cpu_time_us":1418,"apply.run_wall_time_us":1418,"num_ops":2,"prepare.queue_time_us":176366,"prepare.run_cpu_time_us":2020,"prepare.run_wall_time_us":38614,"raft.queue_time_us":555,"raft.run_cpu_time_us":637,"raft.run_wall_time_us":638,"replication_time_us":47971,"spinlock_wait_cycles":47360}]]}
W20260809 01:45:59.613659 10808 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.10.8.125:42063, user_credentials={real_user=slave}} blocked reactor thread for 71589.1us
I20260809 01:45:59.631403 13168 raft_consensus.cc:493] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:59.631843 13168 raft_consensus.cc:515] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.633852 13168 leader_election.cc:290] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:59.638339 11063 leader_election.cc:343] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 761370ec8b164fd184a2e091030f610b
I20260809 01:45:59.638705 11063 leader_election.cc:304] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:59.639585 13179 raft_consensus.cc:2749] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:59.641057 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e03033672e947eab8cc143263281133" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:45:59.641716 10908 raft_consensus.cc:2468] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:45:59.643085 11063 leader_election.cc:304] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:45:59.643849 13168 raft_consensus.cc:2804] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:45:59.644150 13168 raft_consensus.cc:493] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:45:59.644446 13168 raft_consensus.cc:3060] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:45:59.619957 10804 leader_election.cc:343] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:59.622817 10858 consensus_meta.cc:287] T 8bcd72c7dafa49ecbf4183775be51618 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent flushing consensus metadata: real 1.276s user 0.009s sys 0.002s
I20260809 01:45:59.650988 10858 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45982 (request call id 354) took 1539 ms. Trace:
I20260809 01:45:59.651407 10858 rpcz_store.cc:276] 0809 01:45:58.111138 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.169617 (+ 58479us) service_pool.cc:224] Handling call
0809 01:45:58.235178 (+ 65561us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:58.235360 (+ 182us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:58.346000 (+110640us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:59.650120 (+1304120us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:59.650911 (+ 791us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:59.565297 13206 leader_election.cc:290] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:59.621196 13083 consensus_queue.cc:1048] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 01:45:59.662652 11551 tablet_bootstrap.cc:492] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:45:59.672107 10588 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet cfb9afd3be194830ba99f5e20fdaca0a (table default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]). Expected version 1 got 0
I20260809 01:45:59.672989 10588 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet a04881d1c5254053bef50dd3d9d475f2 (table default.Table_57234dd818414b7397d1ac621bdb06a7 [id=17cbc8bf77e142d08d03bd7a051e00ad]). Expected version 2 got 0
I20260809 01:45:59.674567 13179 consensus_queue.cc:1048] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:45:59.675199 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_9c355bbef601486091e79dd70d877f1e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:59.676770 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c355bbef601486091e79dd70d877f1e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:59.678267 12904 ts_tablet_manager.cc:1929] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:45:59.678613 10597 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_25d50c77e3f5425888afe07872b75188" } modify_external_catalogs: true
I20260809 01:45:59.679215 10597 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_25d50c77e3f5425888afe07872b75188" } modify_external_catalogs: true
I20260809 01:45:59.681718 10596 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_12598d7dd6e4411793b992a49dbf090a" } new_table_name: "default.Table_597725134a0749bcb240facde21e0258" modify_external_catalogs: true
I20260809 01:45:59.682451 12904 log.cc:1199] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/c3bf43978dc94c2b9a7f3c18a4677c40
I20260809 01:45:59.690073 12904 ts_tablet_manager.cc:1950] T c3bf43978dc94c2b9a7f3c18a4677c40 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:45:59.686661 11551 tablet_bootstrap.cc:654] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:59.697836 10845 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:45966 (request call id 53) took 1414 ms. Trace:
I20260809 01:45:59.698196 10845 rpcz_store.cc:276] 0809 01:45:58.283429 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.283567 (+ 138us) service_pool.cc:224] Handling call
0809 01:45:59.697636 (+1414069us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:59.699465 12904 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 375) took 1133 ms. Trace:
I20260809 01:45:59.701464 12904 rpcz_store.cc:276] 0809 01:45:58.565764 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.604812 (+ 39048us) service_pool.cc:224] Handling call
0809 01:45:59.416117 (+811305us) ts_tablet_manager.cc:1151] Deleting tablet c3bf43978dc94c2b9a7f3c18a4677c40
0809 01:45:59.416159 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:59.617428 (+201269us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:59.678102 (+ 60674us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:59.699412 (+ 21310us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":810575}
I20260809 01:45:59.704156 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet c3bf43978dc94c2b9a7f3c18a4677c40 (table default.Table_022d458d8ac1457c8e363dfc40bd6880 [id=97f7c7d3e84a4032887e35cf5ea63006]) successfully deleted
I20260809 01:45:59.711925 13206 raft_consensus.cc:493] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:45:59.712316 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet f4985b1715f64adaae432d46dee4ea38 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f4985b1715f64adaae432d46dee4ea38 already in progress: creating tablet
I20260809 01:45:59.712473 13206 raft_consensus.cc:515] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:59.714013 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet d18d1f8343ed4c33a70fccbecc3012a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d18d1f8343ed4c33a70fccbecc3012a6 already in progress: creating tablet
I20260809 01:45:59.714551 13206 leader_election.cc:290] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:59.715999 13206 raft_consensus.cc:493] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:59.716490 13206 raft_consensus.cc:515] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.719696 10588 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet e3684fa56f9549f492a96a5c765eda78 (table default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]). Expected version 1 got 0
I20260809 01:45:59.720963 10588 catalog_manager.cc:5719] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65). New cstate: current_term: 1 leader_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: HEALTHY } } }
I20260809 01:45:59.722420 10588 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 68f022e6a57a4ec0a5a8c33159e97a8e (table default.Table_bbf8548d28c748e19d427c4f7f79622f [id=7e63b2e8084c4dc9b53d011ba1d822e6]). Expected version 1 got 0
I20260809 01:45:59.722935 10588 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 00c7ec3928224059ab7f58d203224536 (table default.Table_4f75e1e877e849fc99c4c673e332c032 [id=374e6a771a36443e9ecf07563041b587]). Expected version 1 got 0
I20260809 01:45:59.723995 10595 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_ca5dbb738ded4d3494afe46eb820bea1 [id=ac1fb532e0824edaa1a3abc07758fbbe] alter complete (version 3)
I20260809 01:45:59.726060 13168 raft_consensus.cc:515] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.738305 10598 master_service.cc:960] ReplaceTablet: received request to replace tablet fa29cba15cc247debfb8e1907ecd0412 from {username='slave'} at 127.0.0.1:37956
I20260809 01:45:59.745335 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14b1d03085eb4ab69789fed6fab882ae" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:59.529038 13161 rpcz_store.cc:276] 0809 01:45:58.305146 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.305440 (+ 294us) service_pool.cc:224] Handling call
0809 01:45:58.609716 (+304276us) ts_tablet_manager.cc:1151] Deleting tablet c528b45de0d14c19af5d6836b7a14f1c
0809 01:45:58.609764 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:59.311140 (+701376us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:59.521323 (+210183us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:59.528652 (+ 7329us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":259908,"thread_start_us":198460,"threads_started":1}
I20260809 01:45:59.635164 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e03033672e947eab8cc143263281133" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:59.729257 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet 403cb09c1786491ab0d40502038bf9c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:43 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:59.729707 11341 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 with delete_type TABLET_DATA_DELETED (replaced by tablet 6542a2b55ce94835a31c732552d87222) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:59.740164 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b02bd8c93b43bf90ff0192a9541aae" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:59.740881 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60176118d61548ff96b001ab0dae7d21" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:59.750965 11062 leader_election.cc:343] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:59.742229 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa12353c25cb4da680d478ce104bc6ba" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:45:59.752302 10591 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_405329d656e44a29bbe12f4ce7752f18" } new_table_name: "default.Table_80bdba205194482d96b26cb139375706" modify_external_catalogs: true
I20260809 01:45:59.753235 13206 leader_election.cc:290] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:45:59.756847 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet c528b45de0d14c19af5d6836b7a14f1c (table default.Table_53624396796f4684acbc3787e6e05f6e [id=18e87d9e1fa3442aaa489e3abcd4640e]) successfully deleted
I20260809 01:45:59.757313 13161 tablet_replica.cc:333] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:45:59.758275 13161 raft_consensus.cc:2243] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:45:59.759270 13161 pending_rounds.cc:70] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316: Trying to abort 1 pending ops.
I20260809 01:45:59.759581 13161 pending_rounds.cc:77] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438862898143232 op_type: NO_OP noop_request { }
W20260809 01:45:59.762818 10805 leader_election.cc:343] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:59.763863 13161 raft_consensus.cc:2889] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
W20260809 01:45:59.765077 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet f4985b1715f64adaae432d46dee4ea38 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f4985b1715f64adaae432d46dee4ea38 already in progress: creating tablet
I20260809 01:45:59.766075 13161 raft_consensus.cc:2272] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 01:45:59.767652 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet d18d1f8343ed4c33a70fccbecc3012a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d18d1f8343ed4c33a70fccbecc3012a6 already in progress: creating tablet
I20260809 01:45:59.766924 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14b1d03085eb4ab69789fed6fab882ae" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:59.767818 11289 leader_election.cc:343] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 403cb09c1786491ab0d40502038bf9c0
I20260809 01:45:59.770572 11289 leader_election.cc:304] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:59.770934 13161 ts_tablet_manager.cc:1916] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:45:59.771517 11289 leader_election.cc:343] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 1bc7dc3dfc2f4c1286ca24b8e3b20719
I20260809 01:45:59.771838 11289 leader_election.cc:304] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:59.772742 13114 raft_consensus.cc:2749] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 01:45:59.773300 13114 raft_consensus.cc:2749] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:59.775296 11551 tablet_bootstrap.cc:492] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:45:59.775774 11551 ts_tablet_manager.cc:1403] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.113s user 0.009s sys 0.008s
W20260809 01:45:59.777856 10596 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa29cba15cc247debfb8e1907ecd0412 consensus state has no leader
I20260809 01:45:59.778330 11551 raft_consensus.cc:359] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.779223 11551 raft_consensus.cc:385] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:59.779511 11551 raft_consensus.cc:740] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:45:59.779958 13168 leader_election.cc:290] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:45:59.780337 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e03033672e947eab8cc143263281133" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:45:59.780119 11551 consensus_queue.cc:260] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.781977 11551 ts_tablet_manager.cc:1434] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 01:45:59.782558 10805 leader_election.cc:304] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:59.783345 13206 raft_consensus.cc:2749] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:59.784052 10804 leader_election.cc:343] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:59.788311 10855 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45982 (request call id 356) took 1650 ms. Trace:
I20260809 01:45:59.788647 10855 rpcz_store.cc:276] 0809 01:45:58.137622 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.314848 (+177226us) service_pool.cc:224] Handling call
0809 01:45:58.316064 (+ 1216us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:45:58.316133 (+ 69us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:45:59.580810 (+1264677us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:59.734052 (+153242us) tablet_replica.cc:206] Creating consensus instance
0809 01:45:59.788100 (+ 54048us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:45:59.797078 10597 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 14b1d03085eb4ab69789fed6fab882ae
I20260809 01:45:59.799965 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e03033672e947eab8cc143263281133" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:45:59.800518 10908 raft_consensus.cc:3060] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:45:59.800715 11341 tablet_service.cc:1558] Processing DeleteTablet for tablet 14b1d03085eb4ab69789fed6fab882ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:59 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:45:59.800630 10855 tablet_service.cc:1558] Processing DeleteTablet for tablet 14b1d03085eb4ab69789fed6fab882ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:59 UTC) from {username='slave'} at 127.0.0.1:45982
W20260809 01:45:59.796748 10805 leader_election.cc:343] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:59.803380 10805 leader_election.cc:304] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:59.804122 11554 tablet_bootstrap.cc:654] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:59.806286 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_ae6c8e43efc446c2887d20c55c3f6fe3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:45:59.807698 13206 raft_consensus.cc:2749] T 60176118d61548ff96b001ab0dae7d21 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:59.807823 10805 leader_election.cc:343] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:45:59.811674 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae6c8e43efc446c2887d20c55c3f6fe3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:59.815086 10805 leader_election.cc:304] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:59.819325 10805 leader_election.cc:343] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:59.819737 10805 leader_election.cc:304] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:45:59.783159 11062 leader_election.cc:343] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:59.799973 11119 tablet_service.cc:1558] Processing DeleteTablet for tablet 14b1d03085eb4ab69789fed6fab882ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:59 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:45:59.822011 11551 tablet_bootstrap.cc:492] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
W20260809 01:45:59.822599 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 14b1d03085eb4ab69789fed6fab882ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14b1d03085eb4ab69789fed6fab882ae already in progress: creating tablet
W20260809 01:45:59.822840 10581 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet fa29cba15cc247debfb8e1907ecd0412 (table default.Table_597725134a0749bcb240facde21e0258 [id=39192535d7454f05b339b6a465b52917], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet fa29cba15cc247debfb8e1907ecd0412 consensus state has no leader
I20260809 01:45:59.826776 13216 tablet_replica.cc:333] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:45:59.827138 11554 tablet_bootstrap.cc:492] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:45:59.827548 11554 ts_tablet_manager.cc:1403] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.271s user 0.009s sys 0.000s
I20260809 01:45:59.827926 13216 raft_consensus.cc:2243] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260809 01:45:59.828404 10591 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1c5897b881c448dab0cafe9238fa4bbb consensus state has no leader
I20260809 01:45:59.828889 13216 raft_consensus.cc:2272] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:45:59.817811 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7770aee1b7af415d995f63672fd05c01" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:45:59.832989 10591 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_3aab7a7876fe46caadaed8dde7387416" } modify_external_catalogs: true
I20260809 01:45:59.833472 10591 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_3aab7a7876fe46caadaed8dde7387416" } modify_external_catalogs: true
W20260809 01:45:59.841681 11062 consensus_peers.cc:597] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:59.849409 13139 raft_consensus.cc:493] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:45:59.849915 13139 raft_consensus.cc:515] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:45:59.838639 10804 leader_election.cc:343] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:59.854038 13216 ts_tablet_manager.cc:1916] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:45:59.854112 13139 leader_election.cc:290] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:45:59.854776 10282 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 135877us
W20260809 01:45:59.855508 11283 leader_election.cc:343] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: 91941616dfdd4df3aaa7c8460876d273
I20260809 01:45:59.866356 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_050ffc9040064a909b37534a21ce9876"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:59.868119 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_050ffc9040064a909b37534a21ce9876 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:59.875638 11551 tablet_bootstrap.cc:654] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:59.879755 11554 raft_consensus.cc:359] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.886669 11554 raft_consensus.cc:385] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:45:59.887019 11554 raft_consensus.cc:740] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:45:59.887852 11553 tablet_bootstrap.cc:654] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:45:59.887681 11554 consensus_queue.cc:260] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.889919 13206 raft_consensus.cc:2749] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:59.892045 10805 raft_consensus.cc:2686] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260809 01:45:59.892550 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 73250.5us
I20260809 01:45:59.897965 10599 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet e3684fa56f9549f492a96a5c765eda78 (table default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]). Expected version 1 got 0
I20260809 01:45:59.899030 10599 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 3e03033672e947eab8cc143263281133 (table default.Table_109c891eea264584b9e1ceab647f32aa [id=24d2e675d969421fba4647380880f606]). Expected version 3 got 0
I20260809 01:45:59.899600 10599 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 63c1a610663a4a2fa58f6cdc5f8d86ce (table default.Table_bd6eedffdf6440efa3e776b7079015d8 [id=8a2084683a7740db80e2c52d3f4c9506]). Expected version 1 got 0
I20260809 01:45:59.915789 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7770aee1b7af415d995f63672fd05c01" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:45:59.924952 11289 leader_election.cc:343] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 91941616dfdd4df3aaa7c8460876d273
I20260809 01:45:59.925478 11289 leader_election.cc:304] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:45:59.926329 11554 ts_tablet_manager.cc:1434] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.099s user 0.008s sys 0.000s
I20260809 01:45:59.927551 13114 raft_consensus.cc:2749] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:45:59.930970 10805 leader_election.cc:343] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:45:59.931452 10805 leader_election.cc:304] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:59.932222 13208 raft_consensus.cc:2749] T 7770aee1b7af415d995f63672fd05c01 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:45:59.932574 13161 ts_tablet_manager.cc:1929] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:45:59.940934 11554 tablet_bootstrap.cc:492] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:45:59.914529 10908 raft_consensus.cc:2468] T 3e03033672e947eab8cc143263281133 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:45:59.956408 11063 leader_election.cc:304] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:45:59.958755 13169 raft_consensus.cc:2804] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:45:59.959286 13169 raft_consensus.cc:697] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:45:59.960196 13169 consensus_queue.cc:237] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:45:59.961259 13161 log.cc:1199] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/7803a6aff02d42c1833f9b2cd7332b9f
I20260809 01:45:59.964505 10598 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet fa29cba15cc247debfb8e1907ecd0412
I20260809 01:45:59.965166 13161 ts_tablet_manager.cc:1950] T 7803a6aff02d42c1833f9b2cd7332b9f P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
W20260809 01:45:59.965425 10805 consensus_peers.cc:597] T 33a5ef5a3ff44f74ab19998ba9f19310 P b5117038a52a45b0aea3d30ca9cfcbe8 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:45:59.967530 11119 tablet_service.cc:1558] Processing DeleteTablet for tablet fa29cba15cc247debfb8e1907ecd0412 with delete_type TABLET_DATA_DELETED (replaced by tablet 023d212fd21643caae5707984467c343) from {username='slave'} at 127.0.0.1:56566
W20260809 01:45:59.969985 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet fa29cba15cc247debfb8e1907ecd0412 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa29cba15cc247debfb8e1907ecd0412 already in progress: creating tablet
I20260809 01:45:59.975950 11553 tablet_bootstrap.cc:492] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:45:59.976538 11553 ts_tablet_manager.cc:1403] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.970s user 0.014s sys 0.005s
I20260809 01:45:59.977897 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_77debdadf8e346f388aac796f7d2cd09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:45:59.979449 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_77debdadf8e346f388aac796f7d2cd09 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:45:59.982105 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_a5f25d7531bb41e5abbe7be00eb9833a" } new_table_name: "default.Table_dfd9db20aa1a4d039db1128f13e1a6ff" modify_external_catalogs: true
I20260809 01:46:00.006340 10597 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_ae40e284667c40089594f8328c9630d7" } modify_external_catalogs: true
I20260809 01:46:00.006856 10597 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_ae40e284667c40089594f8328c9630d7" } modify_external_catalogs: true
I20260809 01:46:00.008452 11551 tablet_bootstrap.cc:492] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:46:00.009025 11551 ts_tablet_manager.cc:1403] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.187s user 0.015s sys 0.004s
I20260809 01:46:00.012100 11551 raft_consensus.cc:359] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.012986 11551 raft_consensus.cc:385] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:00.020401 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_6849525ffb55409998be560b07957dfe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:00.013356 11551 raft_consensus.cc:740] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:46:00.021081 10855 tablet_service.cc:1558] Processing DeleteTablet for tablet fa29cba15cc247debfb8e1907ecd0412 with delete_type TABLET_DATA_DELETED (replaced by tablet 023d212fd21643caae5707984467c343) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:00.021678 10598 catalog_manager.cc:6807] ReplaceTablet: tablet fa29cba15cc247debfb8e1907ecd0412 deleted and replaced by tablet 023d212fd21643caae5707984467c343
W20260809 01:46:00.021888 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6849525ffb55409998be560b07957dfe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:00.022607 11553 raft_consensus.cc:359] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.031744 10598 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_eb6df7e248d44109898ab900441bda8b" } new_table_name: "default.Table_51ec411bde9648a5bce1df1b21c0cf92" modify_external_catalogs: true
I20260809 01:46:00.054665 11551 consensus_queue.cc:260] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.059793 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 362) took 1683 ms. Trace:
I20260809 01:46:00.060199 13161 rpcz_store.cc:276] 0809 01:45:58.376812 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.376967 (+ 155us) service_pool.cc:224] Handling call
0809 01:45:59.752326 (+1375359us) ts_tablet_manager.cc:1151] Deleting tablet 7803a6aff02d42c1833f9b2cd7332b9f
0809 01:45:59.757173 (+ 4847us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:45:59.883242 (+126069us) tablet_metadata.cc:665] Metadata flushed
0809 01:45:59.932452 (+ 49210us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:00.059725 (+127273us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1374450}
I20260809 01:46:00.061391 13161 tablet_replica.cc:333] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:46:00.061941 11554 tablet_bootstrap.cc:654] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:00.062646 13161 raft_consensus.cc:2243] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:00.063248 13161 raft_consensus.cc:2272] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:00.066498 13161 ts_tablet_manager.cc:1916] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:00.077016 11554 tablet_bootstrap.cc:492] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:00.077443 11554 ts_tablet_manager.cc:1403] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.137s user 0.009s sys 0.020s
I20260809 01:46:00.085646 11554 raft_consensus.cc:359] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:46:00.087227 11554 raft_consensus.cc:385] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:00.087481 11554 raft_consensus.cc:740] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:00.088742 10597 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet d9025e61632c461c87972f879db46c87
W20260809 01:46:00.093964 10598 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 74d6b03590c8472db8977bee5962aa6c consensus state has no leader
I20260809 01:46:00.095705 11551 ts_tablet_manager.cc:1434] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.086s user 0.003s sys 0.003s
I20260809 01:46:00.098703 11554 consensus_queue.cc:260] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:46:00.100152 11554 ts_tablet_manager.cc:1434] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.022s user 0.003s sys 0.000s
I20260809 01:46:00.100965 11554 tablet_bootstrap.cc:492] T 3e03033672e947eab8cc143263281133 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:46:00.109381 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 7803a6aff02d42c1833f9b2cd7332b9f (table default.Table_8a405d19ab94416c8d58fff878edae0a [id=42039f2e75f942ac8b96fb47088ad193]) successfully deleted
I20260809 01:46:00.109855 11553 raft_consensus.cc:385] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:00.110237 11553 raft_consensus.cc:740] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:46:00.111063 11553 consensus_queue.cc:260] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:46:00.116111 11289 consensus_peers.cc:597] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b9493892c1ff48e7a0558ff0fcb7a0f3. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:46:00.125808 11341 tablet_service.cc:1558] Processing DeleteTablet for tablet fa29cba15cc247debfb8e1907ecd0412 with delete_type TABLET_DATA_DELETED (replaced by tablet 023d212fd21643caae5707984467c343) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:00.126912 11551 tablet_bootstrap.cc:492] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:46:00.135785 11554 tablet_bootstrap.cc:654] T 3e03033672e947eab8cc143263281133 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:00.142196 11551 tablet_bootstrap.cc:654] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:00.150007 11553 ts_tablet_manager.cc:1434] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.173s user 0.006s sys 0.000s
I20260809 01:46:00.164399 13083 raft_consensus.cc:493] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:00.164940 13083 raft_consensus.cc:515] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.174188 11551 tablet_bootstrap.cc:492] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:46:00.174615 11551 ts_tablet_manager.cc:1403] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.049s user 0.008s sys 0.002s
I20260809 01:46:00.175621 11553 tablet_bootstrap.cc:492] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:46:00.176983 11551 raft_consensus.cc:359] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.177615 11551 raft_consensus.cc:385] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:00.177873 11551 raft_consensus.cc:740] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:46:00.179006 11551 consensus_queue.cc:260] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.180567 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31874b7e70a847a4885bd09da40a6902" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:00.181869 11551 ts_tablet_manager.cc:1434] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
W20260809 01:46:00.181944 10804 leader_election.cc:343] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:00.183521 13083 leader_election.cc:290] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:00.184152 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31874b7e70a847a4885bd09da40a6902" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:00.185659 10805 leader_election.cc:343] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:00.186054 10805 leader_election.cc:304] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:00.189675 13083 raft_consensus.cc:2749] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:46:00.204545 13204 lock_manager.cc:585] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 16400975219178673526) tablet 00000000000000000000000000000000 cur holder 0x7b5800041100
I20260809 01:46:00.207837 11551 tablet_bootstrap.cc:492] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:46:00.213191 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_5a885f5331e5438a9dd35554f82d3d92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:00.214757 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a885f5331e5438a9dd35554f82d3d92 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:00.219704 10598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_6133114bad9b4a68a734ecc6da250f6e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:00.221077 10598 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6133114bad9b4a68a734ecc6da250f6e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:00.224231 11554 tablet_bootstrap.cc:492] T 3e03033672e947eab8cc143263281133 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:00.229507 13216 ts_tablet_manager.cc:1929] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:46:00.230331 11551 tablet_bootstrap.cc:654] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:00.241251 11554 ts_tablet_manager.cc:1403] T 3e03033672e947eab8cc143263281133 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.140s user 0.013s sys 0.013s
I20260809 01:46:00.244468 13161 ts_tablet_manager.cc:1929] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:46:00.244906 13161 log.cc:1199] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/b2726cb09f734ed18bb9379f94f3f22a
I20260809 01:46:00.244848 11554 raft_consensus.cc:359] T 3e03033672e947eab8cc143263281133 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.245573 11554 raft_consensus.cc:385] T 3e03033672e947eab8cc143263281133 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:00.245882 11554 raft_consensus.cc:740] T 3e03033672e947eab8cc143263281133 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:00.253787 11554 consensus_queue.cc:260] T 3e03033672e947eab8cc143263281133 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.252545 11553 tablet_bootstrap.cc:654] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:00.245803 13161 ts_tablet_manager.cc:1950] T b2726cb09f734ed18bb9379f94f3f22a P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:46:00.288900 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_487a0ef0bc0a44cd9ca61c752857b15e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:00.289769 11554 ts_tablet_manager.cc:1434] T 3e03033672e947eab8cc143263281133 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.048s user 0.006s sys 0.000s
W20260809 01:46:00.291484 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_487a0ef0bc0a44cd9ca61c752857b15e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:00.274212 13216 log.cc:1199] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/14b1d03085eb4ab69789fed6fab882ae
I20260809 01:46:00.297497 10596 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_946a0fc83ebc47b99a4d35abd513a966" } new_table_name: "default.Table_2843acdac14b420abec1c3e5e1fbd1b4" modify_external_catalogs: true
I20260809 01:46:00.298569 13216 ts_tablet_manager.cc:1950] T 14b1d03085eb4ab69789fed6fab882ae P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:46:00.301595 10598 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_bbf8548d28c748e19d427c4f7f79622f" } modify_external_catalogs: true
I20260809 01:46:00.302109 10598 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_bbf8548d28c748e19d427c4f7f79622f" } modify_external_catalogs: true
I20260809 01:46:00.302436 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 14b1d03085eb4ab69789fed6fab882ae (table default.Table_25d50c77e3f5425888afe07872b75188 [id=f5a337b5983945e7bc23ddd3818dfff6]) successfully deleted
W20260809 01:46:00.304262 10580 catalog_manager.cc:5030] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet fa29cba15cc247debfb8e1907ecd0412 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa29cba15cc247debfb8e1907ecd0412 already in progress: creating tablet
I20260809 01:46:00.305311 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_a456a4c2d6bc4a26a4badd8bba0504c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:00.306476 11554 tablet_bootstrap.cc:492] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
W20260809 01:46:00.307044 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a456a4c2d6bc4a26a4badd8bba0504c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:46:00.342300 10596 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b38a0a3033cc4328a0200e11c8eb5c31 consensus state has no leader
I20260809 01:46:00.352269 11554 tablet_bootstrap.cc:654] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:00.355491 11551 tablet_bootstrap.cc:492] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:46:00.355885 11551 ts_tablet_manager.cc:1403] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.148s user 0.035s sys 0.020s
I20260809 01:46:00.358172 11551 raft_consensus.cc:359] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.358762 11551 raft_consensus.cc:385] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:00.358991 11551 raft_consensus.cc:740] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:46:00.359550 11551 consensus_queue.cc:260] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.366293 11553 tablet_bootstrap.cc:492] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:46:00.366328 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 363) took 1988 ms. Trace:
I20260809 01:46:00.366863 11553 ts_tablet_manager.cc:1403] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.191s user 0.002s sys 0.010s
I20260809 01:46:00.366705 13161 rpcz_store.cc:276] 0809 01:45:58.378300 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.378432 (+ 132us) service_pool.cc:224] Handling call
0809 01:46:00.061222 (+1682790us) ts_tablet_manager.cc:1151] Deleting tablet b2726cb09f734ed18bb9379f94f3f22a
0809 01:46:00.061269 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:00.148042 (+ 86773us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:00.244346 (+ 96304us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:00.366259 (+121913us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1681869}
I20260809 01:46:00.369472 11553 raft_consensus.cc:359] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.370201 11553 raft_consensus.cc:385] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:00.370517 11553 raft_consensus.cc:740] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:46:00.371255 11553 consensus_queue.cc:260] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.373714 11553 ts_tablet_manager.cc:1434] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20260809 01:46:00.375279 11551 ts_tablet_manager.cc:1434] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.019s user 0.005s sys 0.001s
I20260809 01:46:00.376140 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet b2726cb09f734ed18bb9379f94f3f22a (table default.Table_03cfb94f08614dc5bcdaf9eb182adc1b [id=d23f238a284a47f2be2faf340b853440]) successfully deleted
I20260809 01:46:00.377307 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 364) took 1997 ms. Trace:
I20260809 01:46:00.377672 13161 rpcz_store.cc:276] 0809 01:45:58.379794 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.379941 (+ 147us) service_pool.cc:224] Handling call
0809 01:46:00.376902 (+1996961us) ts_tablet_manager.cc:1151] Deleting tablet c704d148da8d406aa824253a51b850c8
0809 01:46:00.376956 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:00.377250 (+ 294us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1996291}
W20260809 01:46:00.379101 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet c704d148da8d406aa824253a51b850c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c704d148da8d406aa824253a51b850c8 already in progress: creating tablet
I20260809 01:46:00.390316 11551 tablet_bootstrap.cc:492] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:46:00.399160 11554 tablet_bootstrap.cc:492] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:00.399595 11554 ts_tablet_manager.cc:1403] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.093s user 0.008s sys 0.002s
W20260809 01:46:00.400990 10580 catalog_manager.cc:5123] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): alter failed for tablet 7706838afc9f4dbebc3213b82c4da126 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]),no further retry: Not found: Tablet not found: 7706838afc9f4dbebc3213b82c4da126
I20260809 01:46:00.402091 11554 raft_consensus.cc:359] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.402947 11554 raft_consensus.cc:385] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:00.403252 11554 raft_consensus.cc:740] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:00.404134 11554 consensus_queue.cc:260] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.417654 13237 raft_consensus.cc:493] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33b5652538e64ed58a09993d3188f8d7)
I20260809 01:46:00.428673 13237 raft_consensus.cc:515] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.430722 13237 leader_election.cc:290] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:46:00.433032 11289 leader_election.cc:343] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: b6159d660d544b3e907a08372027c0e2
W20260809 01:46:00.435488 11283 leader_election.cc:343] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: b6159d660d544b3e907a08372027c0e2
I20260809 01:46:00.435829 11283 leader_election.cc:304] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:00.436863 13237 raft_consensus.cc:2749] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:46:00.410849 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 370) took 1949 ms. Trace:
I20260809 01:46:00.441874 13161 rpcz_store.cc:276] 0809 01:45:58.461826 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.462048 (+ 222us) service_pool.cc:224] Handling call
0809 01:46:00.410338 (+1948290us) ts_tablet_manager.cc:1151] Deleting tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b
0809 01:46:00.410412 (+ 74us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:00.410768 (+ 356us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1946970}
W20260809 01:46:00.443269 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ad9cdaa5c0a4dc7bfc34db8f18c2b6b already in progress: creating tablet
I20260809 01:46:00.444263 13161 tablet_replica.cc:333] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:46:00.445075 13161 raft_consensus.cc:2243] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:00.445636 13161 raft_consensus.cc:2272] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:00.447028 11551 tablet_bootstrap.cc:654] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:00.447598 13161 ts_tablet_manager.cc:1916] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:00.437575 13083 raft_consensus.cc:493] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:00.456202 13083 raft_consensus.cc:515] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.456851 11553 tablet_bootstrap.cc:492] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:46:00.458326 13083 leader_election.cc:290] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:00.460031 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b31998b65e4264b16c0d73dbcfc73b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:00.462014 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b31998b65e4264b16c0d73dbcfc73b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:00.464349 10804 leader_election.cc:343] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:46:00.466696 10805 leader_election.cc:343] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:00.467075 10805 leader_election.cc:304] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:00.468567 13083 raft_consensus.cc:2749] T 67b31998b65e4264b16c0d73dbcfc73b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:00.479488 13237 raft_consensus.cc:493] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:00.479948 13237 raft_consensus.cc:515] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.482311 13237 leader_election.cc:290] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:00.483939 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0354e4c7bf354f22a9c3c389e0e8ec92" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:00.484302 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0354e4c7bf354f22a9c3c389e0e8ec92" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:46:00.484522 11156 raft_consensus.cc:2468] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:46:00.484851 10908 raft_consensus.cc:2468] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:46:00.486033 11283 leader_election.cc:304] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:46:00.485857 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_5415a483bead450ead7429146a04fb93"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:00.487347 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5415a483bead450ead7429146a04fb93 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:00.487485 13237 raft_consensus.cc:2804] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:00.487849 13237 raft_consensus.cc:493] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:00.488240 13237 raft_consensus.cc:3060] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:00.491343 10592 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_76b0132a831f4433b712389f168fde06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:00.492748 10592 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_76b0132a831f4433b712389f168fde06 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:00.513829 13237 raft_consensus.cc:515] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.523761 11554 ts_tablet_manager.cc:1434] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.124s user 0.006s sys 0.001s
I20260809 01:46:00.524304 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0354e4c7bf354f22a9c3c389e0e8ec92" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:46:00.524926 11149 raft_consensus.cc:3060] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:00.532541 13083 raft_consensus.cc:493] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:00.532989 13083 raft_consensus.cc:515] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.535669 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e360e79820d148f4a6e9fdeb2c19da54" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:00.536056 13083 leader_election.cc:290] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:46:00.537168 10804 leader_election.cc:343] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:00.538764 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e360e79820d148f4a6e9fdeb2c19da54" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:00.540305 10805 leader_election.cc:343] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:00.540694 10805 leader_election.cc:304] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:00.541366 13083 raft_consensus.cc:2749] T e360e79820d148f4a6e9fdeb2c19da54 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:00.546093 11554 tablet_bootstrap.cc:492] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:46:00.550508 13169 raft_consensus.cc:493] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:00.550961 13169 raft_consensus.cc:515] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.556542 10592 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4" } new_table_name: "default.Table_666b764e538240eabf2b6da622d0832f" modify_external_catalogs: true
I20260809 01:46:00.562004 13237 leader_election.cc:290] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:00.563678 13169 leader_election.cc:290] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:46:00.565011 11063 leader_election.cc:343] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 977694496e014ef29f5d7d513e5219c0
I20260809 01:46:00.568190 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_1441945ab21646eba550ae9145deb4fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:00.575536 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977694496e014ef29f5d7d513e5219c0" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:00.581944 11062 leader_election.cc:343] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:00.582392 11062 leader_election.cc:304] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:00.583390 13169 raft_consensus.cc:2749] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:00.588687 11551 tablet_bootstrap.cc:492] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:46:00.589141 11551 ts_tablet_manager.cc:1403] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.199s user 0.009s sys 0.002s
I20260809 01:46:00.589638 11553 tablet_bootstrap.cc:654] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:00.591542 11551 raft_consensus.cc:359] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.592185 11551 raft_consensus.cc:385] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:00.592471 11551 raft_consensus.cc:740] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:46:00.595728 11149 raft_consensus.cc:2468] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:46:00.572180 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0354e4c7bf354f22a9c3c389e0e8ec92" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:46:00.599385 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_bb09705b708949a8a4600ab12d1d7704"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:00.598596 10908 raft_consensus.cc:3060] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:00.605943 13083 raft_consensus.cc:493] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:00.606387 13083 raft_consensus.cc:515] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.608398 13083 leader_election.cc:290] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:00.610849 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9a0fc14dd684ccaa19d51261b9c610e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:00.612350 10805 leader_election.cc:343] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:00.614460 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9a0fc14dd684ccaa19d51261b9c610e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:46:00.619870 10804 leader_election.cc:343] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:00.620232 10804 leader_election.cc:304] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:00.621306 13083 raft_consensus.cc:2749] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:00.623023 11283 leader_election.cc:304] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:46:00.623940 13259 raft_consensus.cc:2804] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:00.624168 11554 tablet_bootstrap.cc:654] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:00.624511 13259 raft_consensus.cc:697] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:46:00.625456 13259 consensus_queue.cc:237] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:46:00.632433 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb09705b708949a8a4600ab12d1d7704 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:00.593060 11551 consensus_queue.cc:260] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:46:00.641213 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1441945ab21646eba550ae9145deb4fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:00.645977 11239 heartbeater.cc:344] Connected to a master server at 127.10.8.125:42063
I20260809 01:46:00.651298 11551 ts_tablet_manager.cc:1434] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.062s user 0.002s sys 0.003s
I20260809 01:46:00.652463 11551 tablet_bootstrap.cc:492] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:46:00.658888 13260 raft_consensus.cc:493] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:00.659500 13260 raft_consensus.cc:515] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.663130 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd09bea5263344919ce26593fe958547" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:00.663926 11149 raft_consensus.cc:2468] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:46:00.663856 11553 tablet_bootstrap.cc:492] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:46:00.664745 11553 ts_tablet_manager.cc:1403] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.208s user 0.010s sys 0.000s
W20260809 01:46:00.664708 11289 leader_election.cc:343] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: cd09bea5263344919ce26593fe958547
I20260809 01:46:00.665457 13260 leader_election.cc:290] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:00.665656 11283 leader_election.cc:304] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:00.668752 13260 raft_consensus.cc:2804] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:00.673846 13260 raft_consensus.cc:493] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:00.674242 13260 raft_consensus.cc:3060] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:00.683308 11553 raft_consensus.cc:359] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:46:00.683943 11553 raft_consensus.cc:385] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:00.684191 11553 raft_consensus.cc:740] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:46:00.687498 10596 master_service.cc:960] ReplaceTablet: received request to replace tablet 31874b7e70a847a4885bd09da40a6902 from {username='slave'} at 127.0.0.1:37956
I20260809 01:46:00.693993 11551 tablet_bootstrap.cc:654] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:00.707715 10908 raft_consensus.cc:2468] T 0354e4c7bf354f22a9c3c389e0e8ec92 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:46:00.717759 11387 raft_consensus.cc:3060] T 3e03033672e947eab8cc143263281133 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:00.684901 11553 consensus_queue.cc:260] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:46:00.721084 11553 ts_tablet_manager.cc:1434] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.056s user 0.005s sys 0.000s
I20260809 01:46:00.721560 13260 raft_consensus.cc:515] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.724140 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd09bea5263344919ce26593fe958547" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:46:00.724733 11149 raft_consensus.cc:3060] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:00.731832 11149 raft_consensus.cc:2468] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:46:00.731762 13264 consensus_queue.cc:1048] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:46:00.733158 11283 leader_election.cc:304] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:46:00.733923 13237 raft_consensus.cc:2804] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:00.742386 13260 leader_election.cc:290] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:46:00.746997 11289 leader_election.cc:343] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: cd09bea5263344919ce26593fe958547
I20260809 01:46:00.746978 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_4e15d84c0f2c4968878a65905971141b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:00.748910 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4e15d84c0f2c4968878a65905971141b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:00.751549 13237 raft_consensus.cc:697] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:46:00.752422 13237 consensus_queue.cc:237] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.764837 11551 tablet_bootstrap.cc:492] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:46:00.765393 11551 ts_tablet_manager.cc:1403] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.113s user 0.014s sys 0.000s
I20260809 01:46:00.767730 13259 raft_consensus.cc:493] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:00.768498 13259 raft_consensus.cc:515] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:46:00.773231 11289 leader_election.cc:343] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: a38bd344108649ed8e1687f82154000b
I20260809 01:46:00.775964 11551 raft_consensus.cc:359] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
W20260809 01:46:00.780892 11283 leader_election.cc:343] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: a38bd344108649ed8e1687f82154000b
I20260809 01:46:00.781260 11283 leader_election.cc:304] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:00.782018 13273 raft_consensus.cc:2749] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:00.783967 13259 leader_election.cc:290] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:46:00.787093 11554 tablet_bootstrap.cc:492] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:00.787509 11554 ts_tablet_manager.cc:1403] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.242s user 0.010s sys 0.000s
I20260809 01:46:00.790541 11554 raft_consensus.cc:359] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.792165 11554 raft_consensus.cc:385] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:00.792418 11554 raft_consensus.cc:740] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:00.793049 11551 raft_consensus.cc:385] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:00.793329 11551 raft_consensus.cc:740] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:46:00.794313 11554 consensus_queue.cc:260] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.795817 11551 consensus_queue.cc:260] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:46:00.799757 11554 ts_tablet_manager.cc:1434] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260809 01:46:00.800693 11554 tablet_bootstrap.cc:492] T 2692a931500443d298e25a2b362b22a1 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:46:00.806950 11551 ts_tablet_manager.cc:1434] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.041s user 0.007s sys 0.000s
I20260809 01:46:00.816396 11554 tablet_bootstrap.cc:654] T 2692a931500443d298e25a2b362b22a1 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:00.818341 11553 tablet_bootstrap.cc:492] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
W20260809 01:46:00.824043 10580 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 1c5897b881c448dab0cafe9238fa4bbb (table default.Table_80bdba205194482d96b26cb139375706 [id=b098ff5e09a0421ca14522917ab6e4f1], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1c5897b881c448dab0cafe9238fa4bbb consensus state has no leader
I20260809 01:46:00.853304 13237 raft_consensus.cc:493] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:00.853863 13237 raft_consensus.cc:515] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.856081 13237 leader_election.cc:290] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:46:00.868027 11289 leader_election.cc:343] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 761370ec8b164fd184a2e091030f610b
W20260809 01:46:00.875159 10282 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 77421.4us
I20260809 01:46:00.882663 13161 ts_tablet_manager.cc:1929] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:46:00.883157 13161 log.cc:1199] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/b6159d660d544b3e907a08372027c0e2
I20260809 01:46:00.883970 13161 ts_tablet_manager.cc:1950] T b6159d660d544b3e907a08372027c0e2 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:46:00.886475 13264 consensus_queue.cc:1048] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:46:00.891520 11553 tablet_bootstrap.cc:654] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:00.894866 13083 raft_consensus.cc:493] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:00.895381 13083 raft_consensus.cc:515] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:46:00.899802 13083 leader_election.cc:290] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:00.906066 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04881d1c5254053bef50dd3d9d475f2" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:00.908010 10805 leader_election.cc:343] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:00.915172 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04881d1c5254053bef50dd3d9d475f2" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:00.920078 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761370ec8b164fd184a2e091030f610b" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:00.920387 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 371) took 2456 ms. Trace:
I20260809 01:46:00.920962 11149 raft_consensus.cc:2468] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:46:00.920781 13161 rpcz_store.cc:276] 0809 01:45:58.463784 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.463900 (+ 116us) service_pool.cc:224] Handling call
0809 01:46:00.444108 (+1980208us) ts_tablet_manager.cc:1151] Deleting tablet b6159d660d544b3e907a08372027c0e2
0809 01:46:00.444152 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:00.675670 (+231518us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:00.882519 (+206849us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:00.920307 (+ 37788us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1979519}
W20260809 01:46:00.922933 10804 leader_election.cc:343] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:00.923319 10804 leader_election.cc:304] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:00.926397 11283 leader_election.cc:304] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:00.927546 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet b6159d660d544b3e907a08372027c0e2 (table default.Table_99bad24613e94b6f840f637bea4978fc [id=ddfe680283854b9484dc3e9d6e626ab9]) successfully deleted
I20260809 01:46:00.928026 13273 raft_consensus.cc:2804] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:00.928339 13273 raft_consensus.cc:493] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:00.928633 13273 raft_consensus.cc:3060] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:00.944978 11551 tablet_bootstrap.cc:492] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:46:00.945948 13273 raft_consensus.cc:515] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.951879 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761370ec8b164fd184a2e091030f610b" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:46:00.952533 11149 raft_consensus.cc:3060] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:00.955709 13273 leader_election.cc:290] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:00.959417 13161 tablet_replica.cc:333] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:46:00.960322 13161 raft_consensus.cc:2243] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:46:00.961216 13161 pending_rounds.cc:70] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316: Trying to abort 1 pending ops.
I20260809 01:46:00.961107 11149 raft_consensus.cc:2468] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:46:00.962497 11283 leader_election.cc:304] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:46:00.965018 11554 tablet_bootstrap.cc:492] T 2692a931500443d298e25a2b362b22a1 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:00.965421 11554 ts_tablet_manager.cc:1403] T 2692a931500443d298e25a2b362b22a1 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.165s user 0.029s sys 0.022s
I20260809 01:46:00.968433 11554 raft_consensus.cc:359] T 2692a931500443d298e25a2b362b22a1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.965129 13273 raft_consensus.cc:2804] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:00.969049 11554 raft_consensus.cc:385] T 2692a931500443d298e25a2b362b22a1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:00.969501 13273 raft_consensus.cc:697] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:46:00.969332 11554 raft_consensus.cc:740] T 2692a931500443d298e25a2b362b22a1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:00.970389 13273 consensus_queue.cc:237] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.971124 11554 consensus_queue.cc:260] T 2692a931500443d298e25a2b362b22a1 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.973469 11554 ts_tablet_manager.cc:1434] T 2692a931500443d298e25a2b362b22a1 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.008s user 0.004s sys 0.001s
I20260809 01:46:00.974195 11551 tablet_bootstrap.cc:654] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:00.978736 11554 tablet_bootstrap.cc:492] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:46:00.987020 11554 tablet_bootstrap.cc:654] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:00.993798 11551 tablet_bootstrap.cc:492] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:46:00.994190 11551 ts_tablet_manager.cc:1403] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.049s user 0.017s sys 0.001s
I20260809 01:46:00.996795 11551 raft_consensus.cc:359] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.997385 11551 raft_consensus.cc:385] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:00.997653 11551 raft_consensus.cc:740] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:46:00.998248 11551 consensus_queue.cc:260] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:00.961459 13161 pending_rounds.cc:77] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316438870652878848 op_type: NO_OP noop_request { }
I20260809 01:46:00.999643 13161 raft_consensus.cc:2889] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 01:46:01.000104 13161 raft_consensus.cc:2272] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:01.020435 11551 ts_tablet_manager.cc:1434] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20260809 01:46:01.032078 11392 raft_consensus.cc:3060] T 2692a931500443d298e25a2b362b22a1 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:01.006673 13083 raft_consensus.cc:2749] T a04881d1c5254053bef50dd3d9d475f2 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:46:01.037184 11289 leader_election.cc:343] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 761370ec8b164fd184a2e091030f610b
I20260809 01:46:01.039358 11551 tablet_bootstrap.cc:492] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:46:01.046139 13161 ts_tablet_manager.cc:1916] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:01.048543 11551 tablet_bootstrap.cc:654] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:01.049666 13289 raft_consensus.cc:493] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:01.050283 13289 raft_consensus.cc:515] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.055156 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8323a7a40a1445f980c0adcf6a6d16c7" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:01.058555 11062 leader_election.cc:343] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:01.037184 11554 tablet_bootstrap.cc:492] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
W20260809 01:46:01.070710 11063 leader_election.cc:343] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 8323a7a40a1445f980c0adcf6a6d16c7
I20260809 01:46:01.071132 11554 ts_tablet_manager.cc:1403] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.093s user 0.008s sys 0.002s
I20260809 01:46:01.071080 11063 leader_election.cc:304] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:01.071930 13169 raft_consensus.cc:2749] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:01.072669 13289 leader_election.cc:290] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:01.084826 11554 raft_consensus.cc:359] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.085547 11554 raft_consensus.cc:385] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:01.085819 11554 raft_consensus.cc:740] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:01.089740 11554 consensus_queue.cc:260] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.091802 11554 ts_tablet_manager.cc:1434] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20260809 01:46:01.092730 11554 tablet_bootstrap.cc:492] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:46:01.130189 11553 tablet_bootstrap.cc:492] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:46:01.130591 11553 ts_tablet_manager.cc:1403] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.313s user 0.009s sys 0.012s
I20260809 01:46:01.132241 11551 tablet_bootstrap.cc:492] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:46:01.132643 11551 ts_tablet_manager.cc:1403] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.094s user 0.009s sys 0.006s
I20260809 01:46:01.133999 11553 raft_consensus.cc:359] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.134665 11553 raft_consensus.cc:385] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:01.136051 11551 raft_consensus.cc:359] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.139717 11551 raft_consensus.cc:385] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:01.140084 11551 raft_consensus.cc:740] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:46:01.137564 13083 raft_consensus.cc:493] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:01.141687 13083 raft_consensus.cc:515] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.141950 13158 raft_consensus.cc:493] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 629f002048f64491afdd57f72a466e59)
I20260809 01:46:01.142689 13158 raft_consensus.cc:515] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:46:01.145646 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e2f77a70794ff7811b5f09d885904b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:46:01.145531 11551 consensus_queue.cc:260] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.145792 13171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 629f002048f64491afdd57f72a466e59. Latest consensus state: current_term: 2 leader_uuid: "629f002048f64491afdd57f72a466e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:46:01.146746 13171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 01:46:01.147746 10805 leader_election.cc:343] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:01.149746 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e2f77a70794ff7811b5f09d885904b" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:01.150405 10356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 453 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
W20260809 01:46:01.151399 10804 leader_election.cc:343] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:01.151724 10804 leader_election.cc:304] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:01.152421 13296 raft_consensus.cc:2749] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:01.153235 13083 leader_election.cc:290] T 02e2f77a70794ff7811b5f09d885904b P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:01.134930 11553 raft_consensus.cc:740] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:46:01.156227 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 453 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:46:01.158110 12870 leader_election.cc:304] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bc72deb1d2402cabd817dfe3b27e88; no voters: 09a73ee8e8ec496a984efcab27715506, 629f002048f64491afdd57f72a466e59
I20260809 01:46:01.159255 13171 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 01:46:01.160552 13158 leader_election.cc:290] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:46:01.162889 11551 ts_tablet_manager.cc:1434] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.030s user 0.003s sys 0.004s
I20260809 01:46:01.165165 13303 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 01:46:01.170214 11551 tablet_bootstrap.cc:492] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:46:01.182060 11554 tablet_bootstrap.cc:654] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:01.202100 13289 raft_consensus.cc:493] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:01.202656 13289 raft_consensus.cc:515] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.204922 13289 leader_election.cc:290] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:01.207389 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7a8b6d06064addb02979a7feedf756" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:01.209117 11062 leader_election.cc:343] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:46:01.210839 11063 leader_election.cc:343] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: bb7a8b6d06064addb02979a7feedf756
I20260809 01:46:01.211264 11063 leader_election.cc:304] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:01.212536 13289 raft_consensus.cc:2749] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:01.213248 12931 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" instance_seqno: 1786239923967594) as {username='slave'} at 127.10.8.66:37605; Asking this server to re-register.
I20260809 01:46:01.215721 11553 consensus_queue.cc:260] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.224165 11551 tablet_bootstrap.cc:654] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:01.230886 11553 ts_tablet_manager.cc:1434] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.100s user 0.007s sys 0.000s
I20260809 01:46:01.232019 11553 tablet_bootstrap.cc:492] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:46:01.217188 12933 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" instance_seqno: 1786239927006341) as {username='slave'} at 127.10.8.67:36721; Asking this server to re-register.
I20260809 01:46:01.270403 13289 raft_consensus.cc:493] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:01.271045 13289 raft_consensus.cc:515] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:46:01.274549 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325e3d19b83545a197c2e09fc9643b5e" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:46:01.275143 10908 raft_consensus.cc:2468] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:46:01.280864 10597 catalog_manager.cc:5719] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67). New cstate: current_term: 1 leader_uuid: "4f60c9ac9414403d9ea5a97b103c2316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } }
I20260809 01:46:01.282651 13169 raft_consensus.cc:493] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:01.283187 13169 raft_consensus.cc:515] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.283041 10597 catalog_manager.cc:5719] T 3e03033672e947eab8cc143263281133 P 4f60c9ac9414403d9ea5a97b103c2316 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66). New cstate: current_term: 1 leader_uuid: "33b5652538e64ed58a09993d3188f8d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } }
I20260809 01:46:01.284332 10597 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 3e03033672e947eab8cc143263281133 (table default.Table_109c891eea264584b9e1ceab647f32aa [id=24d2e675d969421fba4647380880f606]). Expected version 3 got 0
I20260809 01:46:01.282092 13289 leader_election.cc:290] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:01.286736 11063 leader_election.cc:304] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:46:01.288266 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7a511512184f53921cea081f82aa37" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:46:01.288765 10908 raft_consensus.cc:2468] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:46:01.289835 11063 leader_election.cc:304] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:46:01.290661 13264 raft_consensus.cc:2804] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:01.291021 13264 raft_consensus.cc:493] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:01.291375 13264 raft_consensus.cc:3060] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:01.293715 13083 raft_consensus.cc:493] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:01.294293 13083 raft_consensus.cc:515] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:46:01.296981 13083 leader_election.cc:290] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:01.297659 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9465f8bb39d64e1cab8646282cb65c75" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:46:01.299036 10804 leader_election.cc:343] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:01.286736 13169 leader_election.cc:290] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:01.305179 13169 raft_consensus.cc:2804] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:01.306246 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9465f8bb39d64e1cab8646282cb65c75" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:01.307910 10805 leader_election.cc:343] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:01.308277 10805 leader_election.cc:304] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:01.309996 13083 raft_consensus.cc:2749] T 9465f8bb39d64e1cab8646282cb65c75 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:01.310799 11341 tablet_service.cc:1511] Processing CreateTablet for tablet 44440a89626b447d9a2715cba63d9322 (DEFAULT_TABLE table=default.Table_ff631a5d638e4a32a994369553091330 [id=ce472fcb8b284e7f84ef7240d1880e57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.311797 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44440a89626b447d9a2715cba63d9322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.312371 13169 raft_consensus.cc:493] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:01.312693 13169 raft_consensus.cc:3060] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:01.317829 13289 raft_consensus.cc:493] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:01.318943 13289 raft_consensus.cc:515] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.326342 11119 tablet_service.cc:1511] Processing CreateTablet for tablet 44440a89626b447d9a2715cba63d9322 (DEFAULT_TABLE table=default.Table_ff631a5d638e4a32a994369553091330 [id=ce472fcb8b284e7f84ef7240d1880e57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.327991 11119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44440a89626b447d9a2715cba63d9322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.328536 13264 raft_consensus.cc:515] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:46:01.332551 10855 tablet_service.cc:1511] Processing CreateTablet for tablet 44440a89626b447d9a2715cba63d9322 (DEFAULT_TABLE table=default.Table_ff631a5d638e4a32a994369553091330 [id=ce472fcb8b284e7f84ef7240d1880e57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.333921 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44440a89626b447d9a2715cba63d9322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.336668 10589 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet cfb9afd3be194830ba99f5e20fdaca0a (table default.Table_205ca6d8f3a54006a86ff2cb1bffc8d4 [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]). Expected version 1 got 0
I20260809 01:46:01.340162 10589 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 63c1a610663a4a2fa58f6cdc5f8d86ce (table default.Table_bd6eedffdf6440efa3e776b7079015d8 [id=8a2084683a7740db80e2c52d3f4c9506]). Expected version 1 got 0
I20260809 01:46:01.341837 10589 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 60176118d61548ff96b001ab0dae7d21 (table default.Table_3dd6aaf3d0d545529f081db48de38cc2 [id=b90c58e7b4a34ea2a22ec6e5aa4be490]). Expected version 1 got 0
I20260809 01:46:01.346694 11239 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:01.351552 10589 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 00c7ec3928224059ab7f58d203224536 (table default.Table_4f75e1e877e849fc99c4c673e332c032 [id=374e6a771a36443e9ecf07563041b587]). Expected version 1 got 0
I20260809 01:46:01.352157 10589 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet a04881d1c5254053bef50dd3d9d475f2 (table default.Table_57234dd818414b7397d1ac621bdb06a7 [id=17cbc8bf77e142d08d03bd7a051e00ad]). Expected version 2 got 0
I20260809 01:46:01.352597 10589 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet e3684fa56f9549f492a96a5c765eda78 (table default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]). Expected version 1 got 0
I20260809 01:46:01.359754 12931 ts_manager.cc:194] Registered new tserver with Master: 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495)
I20260809 01:46:01.366402 11383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f45c02a6994c5aaaf9c0a499e15fed" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:46:01.370923 10596 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 31874b7e70a847a4885bd09da40a6902
I20260809 01:46:01.374357 10847 tablet_service.cc:1558] Processing DeleteTablet for tablet 31874b7e70a847a4885bd09da40a6902 with delete_type TABLET_DATA_DELETED (replaced by tablet f5158431dbcf4eef91e9f4bc28d826fc) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:01.375382 10591 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 100ad529186e4821b85ba28996933a8a
I20260809 01:46:01.376724 10591 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 461) took 1544 ms. Trace:
I20260809 01:46:01.378458 11361 tablet_service.cc:1511] Processing CreateTablet for tablet f9069292d93e42e0891c6c3e2febc552 (DEFAULT_TABLE table=default.Table_3c13b21368154865a2a3d2b18f7db9ca [id=e0779d9dcb7f4e5295fcd43596189488]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.379506 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9069292d93e42e0891c6c3e2febc552. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.379526 11117 tablet_service.cc:1558] Processing DeleteTablet for tablet 31874b7e70a847a4885bd09da40a6902 with delete_type TABLET_DATA_DELETED (replaced by tablet f5158431dbcf4eef91e9f4bc28d826fc) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:01.380815 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet 100ad529186e4821b85ba28996933a8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:59 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:01.377032 10591 rpcz_store.cc:276] 0809 01:45:59.832604 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:59.832715 (+ 111us) service_pool.cc:224] Handling call
0809 01:45:59.833875 (+ 1160us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:59.835482 (+ 1607us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:45:59.835599 (+ 117us) catalog_manager.cc:2850] Locking tablets
0809 01:45:59.852492 (+ 16893us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:45:59.979973 (+127481us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:45:59.980157 (+ 184us) catalog_manager.cc:2898] Committing in-memory state
0809 01:46:01.375283 (+1395126us) catalog_manager.cc:2902] Aborting table tasks
0809 01:46:01.376669 (+ 1386us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:59.945073 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:59.945489 (+ 416us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:59.945533 (+ 44us) write_op.cc:435] Acquired schema lock
0809 01:45:59.945548 (+ 15us) tablet.cc:662] Decoding operations
0809 01:45:59.945864 (+ 316us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:59.945899 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:59.945918 (+ 19us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:45:59.946195 (+ 277us) tablet.cc:701] Row locks acquired
0809 01:45:59.946208 (+ 13us) write_op.cc:260] PREPARE: finished
0809 01:45:59.946316 (+ 108us) write_op.cc:270] Start()
0809 01:45:59.946388 (+ 72us) write_op.cc:276] Timestamp: P: 1786239959946270 usec, L: 0
0809 01:45:59.946415 (+ 27us) op_driver.cc:348] REPLICATION: starting
0809 01:45:59.946870 (+ 455us) log.cc:844] Serialized 725 byte log entry
0809 01:45:59.976405 (+ 29535us) op_driver.cc:464] REPLICATION: finished
0809 01:45:59.977051 (+ 646us) write_op.cc:301] APPLY: starting
0809 01:45:59.977107 (+ 56us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:59.977212 (+ 105us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:59.977298 (+ 86us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:59.977557 (+ 259us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:59.977699 (+ 142us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:45:59.977741 (+ 42us) write_op.cc:312] APPLY: finished
0809 01:45:59.977940 (+ 199us) log.cc:844] Serialized 29 byte log entry
0809 01:45:59.978078 (+ 138us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:59.978139 (+ 61us) write_op.cc:454] Released schema lock
0809 01:45:59.978172 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:59.978381 (+ 209us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:59.978404 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":455,"apply.run_cpu_time_us":1301,"apply.run_wall_time_us":1300,"num_ops":2,"prepare.queue_time_us":90896,"prepare.run_cpu_time_us":1998,"prepare.run_wall_time_us":1997,"replication_time_us":29861}]]}
I20260809 01:46:01.386329 11351 tablet_service.cc:1558] Processing DeleteTablet for tablet 100ad529186e4821b85ba28996933a8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:59 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:01.386797 10858 tablet_service.cc:1558] Processing DeleteTablet for tablet 100ad529186e4821b85ba28996933a8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:59 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:01.391338 10596 catalog_manager.cc:6807] ReplaceTablet: tablet 31874b7e70a847a4885bd09da40a6902 deleted and replaced by tablet f5158431dbcf4eef91e9f4bc28d826fc
I20260809 01:46:01.392144 13169 raft_consensus.cc:515] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.395961 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_83678aa7b12042f1b89ee51be4cb1d7a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:01.396845 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7a511512184f53921cea081f82aa37" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:46:01.406664 10858 tablet_service.cc:1511] Processing CreateTablet for tablet f9069292d93e42e0891c6c3e2febc552 (DEFAULT_TABLE table=default.Table_3c13b21368154865a2a3d2b18f7db9ca [id=e0779d9dcb7f4e5295fcd43596189488]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.407714 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9069292d93e42e0891c6c3e2febc552. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.397871 13289 leader_election.cc:290] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:46:01.400045 11063 leader_election.cc:343] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 50f45c02a6994c5aaaf9c0a499e15fed
I20260809 01:46:01.404850 11116 tablet_service.cc:1511] Processing CreateTablet for tablet f9069292d93e42e0891c6c3e2febc552 (DEFAULT_TABLE table=default.Table_3c13b21368154865a2a3d2b18f7db9ca [id=e0779d9dcb7f4e5295fcd43596189488]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.412449 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9069292d93e42e0891c6c3e2febc552. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.421380 11553 tablet_bootstrap.cc:654] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:01.426244 11551 tablet_bootstrap.cc:492] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:46:01.426652 11551 ts_tablet_manager.cc:1403] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.257s user 0.015s sys 0.019s
I20260809 01:46:01.426638 11111 tablet_service.cc:1511] Processing CreateTablet for tablet b788296871744cc49967fb00af4ad7a9 (DEFAULT_TABLE table=default.Table_a2532aa2811644d3ae76f4cdc35c0518 [id=dbf6c91b5fd5400f838e23d76af82536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.400650 11336 tablet_service.cc:1558] Processing DeleteTablet for tablet 31874b7e70a847a4885bd09da40a6902 with delete_type TABLET_DATA_DELETED (replaced by tablet f5158431dbcf4eef91e9f4bc28d826fc) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:01.427564 13308 tablet_replica.cc:333] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:46:01.427654 11111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b788296871744cc49967fb00af4ad7a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.402458 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325e3d19b83545a197c2e09fc9643b5e" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:46:01.428794 13308 raft_consensus.cc:2243] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:01.431102 10861 tablet_service.cc:1511] Processing CreateTablet for tablet b788296871744cc49967fb00af4ad7a9 (DEFAULT_TABLE table=default.Table_a2532aa2811644d3ae76f4cdc35c0518 [id=dbf6c91b5fd5400f838e23d76af82536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.432040 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b788296871744cc49967fb00af4ad7a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.435631 11551 raft_consensus.cc:359] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.436267 11551 raft_consensus.cc:385] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:01.436532 11551 raft_consensus.cc:740] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:46:01.437098 11551 consensus_queue.cc:260] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.439018 11551 ts_tablet_manager.cc:1434] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260809 01:46:01.439122 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7a511512184f53921cea081f82aa37" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:46:01.439805 13264 leader_election.cc:290] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:46:01.440819 11062 leader_election.cc:343] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:01.440884 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325e3d19b83545a197c2e09fc9643b5e" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:46:01.441319 10908 raft_consensus.cc:3060] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:01.441211 11062 leader_election.cc:304] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:01.446851 11554 tablet_bootstrap.cc:492] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:01.447320 11554 ts_tablet_manager.cc:1403] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.355s user 0.015s sys 0.006s
I20260809 01:46:01.447525 11465 heartbeater.cc:461] Registering TS with master...
W20260809 01:46:01.450685 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_83678aa7b12042f1b89ee51be4cb1d7a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:01.411130 10592 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet e3684fa56f9549f492a96a5c765eda78 (table default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]). Expected version 1 got 0
I20260809 01:46:01.429317 13308 raft_consensus.cc:2272] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:01.454285 10592 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 325e3d19b83545a197c2e09fc9643b5e (table default.Table_66160146b5684111a0b61024efd70dfb [id=ccd0ea07da604f859047f2eca85fc566]). Expected version 3 got 0
I20260809 01:46:01.455856 13308 ts_tablet_manager.cc:1916] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:01.455650 11336 tablet_service.cc:1511] Processing CreateTablet for tablet b788296871744cc49967fb00af4ad7a9 (DEFAULT_TABLE table=default.Table_a2532aa2811644d3ae76f4cdc35c0518 [id=dbf6c91b5fd5400f838e23d76af82536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.456578 10592 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 63c1a610663a4a2fa58f6cdc5f8d86ce (table default.Table_bd6eedffdf6440efa3e776b7079015d8 [id=8a2084683a7740db80e2c52d3f4c9506]). Expected version 1 got 0
I20260809 01:46:01.456346 11554 raft_consensus.cc:359] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.457026 11554 raft_consensus.cc:385] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:01.457274 11554 raft_consensus.cc:740] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:01.457492 10592 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 3e03033672e947eab8cc143263281133 (table default.Table_109c891eea264584b9e1ceab647f32aa [id=24d2e675d969421fba4647380880f606]). Expected version 3 got 0
I20260809 01:46:01.458185 13161 ts_tablet_manager.cc:1929] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:46:01.458575 13161 log.cc:1199] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/b9493892c1ff48e7a0558ff0fcb7a0f3
W20260809 01:46:01.459677 13262 log.cc:927] Time spent T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: Append to log took a long time: real 0.762s user 0.047s sys 0.007s
I20260809 01:46:01.461239 11336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b788296871744cc49967fb00af4ad7a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.465516 11105 tablet_service.cc:1511] Processing CreateTablet for tablet 438aecb7e5e544b393edbbe5df7966a9 (DEFAULT_TABLE table=default.Table_985df37273284bb7be1a28211209ecfa [id=386aa8599e21406397d1b975dc9b7fd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.466522 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 438aecb7e5e544b393edbbe5df7966a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.466768 13161 ts_tablet_manager.cc:1950] T b9493892c1ff48e7a0558ff0fcb7a0f3 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:46:01.469491 11341 tablet_service.cc:1511] Processing CreateTablet for tablet 438aecb7e5e544b393edbbe5df7966a9 (DEFAULT_TABLE table=default.Table_985df37273284bb7be1a28211209ecfa [id=386aa8599e21406397d1b975dc9b7fd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.472339 13264 raft_consensus.cc:2749] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:01.471262 12932 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" instance_seqno: 1786239921069824) as {username='slave'} at 127.10.8.65:38897; Asking this server to re-register.
W20260809 01:46:01.476902 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 31874b7e70a847a4885bd09da40a6902 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31874b7e70a847a4885bd09da40a6902 already in progress: creating tablet
W20260809 01:46:01.477849 10576 catalog_manager.cc:5030] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): delete failed for tablet 100ad529186e4821b85ba28996933a8a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 100ad529186e4821b85ba28996933a8a already in progress: creating tablet
I20260809 01:46:01.478543 10986 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:01.485321 10908 raft_consensus.cc:2468] T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:46:01.488782 11351 tablet_service.cc:1511] Processing CreateTablet for tablet fcb8b3e19ceb4988aad0ca6b4f6b4514 (DEFAULT_TABLE table=default.Table_190f8292dbc54632996f644b10d5c284 [id=a1015ae30d514882b85e1f704146ebf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.494674 11117 tablet_service.cc:1511] Processing CreateTablet for tablet fcb8b3e19ceb4988aad0ca6b4f6b4514 (DEFAULT_TABLE table=default.Table_190f8292dbc54632996f644b10d5c284 [id=a1015ae30d514882b85e1f704146ebf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.496057 11117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcb8b3e19ceb4988aad0ca6b4f6b4514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.497958 12932 ts_manager.cc:194] Registered new tserver with Master: b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:01.506915 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_32967e2b77ab462caff32362df47f2c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:01.508730 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_32967e2b77ab462caff32362df47f2c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:01.509029 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcb8b3e19ceb4988aad0ca6b4f6b4514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.514091 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 438aecb7e5e544b393edbbe5df7966a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.524575 11345 tablet_service.cc:1511] Processing CreateTablet for tablet 87dfe669489e490ebd431e71964d7754 (DEFAULT_TABLE table=default.Table_a5ecac2f48c1491fb074981ce05ef950 [id=a8acdfdf9aff4072a75d0dc017880b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.526467 11345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87dfe669489e490ebd431e71964d7754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.532892 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet cfb9afd3be194830ba99f5e20fdaca0a (table default.Table_666b764e538240eabf2b6da622d0832f [id=5aa4b9c7be9447a3ba1bfda3a6e689b0]). Expected version 2 got 0
I20260809 01:46:01.533828 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 63c1a610663a4a2fa58f6cdc5f8d86ce (table default.Table_bd6eedffdf6440efa3e776b7079015d8 [id=8a2084683a7740db80e2c52d3f4c9506]). Expected version 1 got 0
I20260809 01:46:01.544523 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 00c7ec3928224059ab7f58d203224536 (table default.Table_4f75e1e877e849fc99c4c673e332c032 [id=374e6a771a36443e9ecf07563041b587]). Expected version 1 got 0
I20260809 01:46:01.545154 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet f9a0fc14dd684ccaa19d51261b9c610e (table default.Table_33b1a3980a3f46b6bd99b4935671527c [id=2993a04ef2744f019c069da7ccc24532]). Expected version 1 got 0
I20260809 01:46:01.545499 11349 tablet_service.cc:1511] Processing CreateTablet for tablet edc25b46fdaf41b3b7ebab1237266b98 (DEFAULT_TABLE table=default.Table_2782952361d24cf9aa19e0667a8a67ee [id=87628394e909476a9dc338f1776a495c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.547152 11349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc25b46fdaf41b3b7ebab1237266b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.546800 11554 consensus_queue.cc:260] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.549243 11554 ts_tablet_manager.cc:1434] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.102s user 0.005s sys 0.000s
I20260809 01:46:01.554210 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet a04881d1c5254053bef50dd3d9d475f2 (table default.Table_57234dd818414b7397d1ac621bdb06a7 [id=17cbc8bf77e142d08d03bd7a051e00ad]). Expected version 2 got 0
I20260809 01:46:01.555001 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet e3684fa56f9549f492a96a5c765eda78 (table default.Table_a5f25d7531bb41e5abbe7be00eb9833a [id=778daa8e84c6441ab0ec13ad54453619]). Expected version 1 got 0
I20260809 01:46:01.565897 11105 tablet_service.cc:1511] Processing CreateTablet for tablet 87dfe669489e490ebd431e71964d7754 (DEFAULT_TABLE table=default.Table_a5ecac2f48c1491fb074981ce05ef950 [id=a8acdfdf9aff4072a75d0dc017880b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.566429 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 384) took 2438 ms. Trace:
I20260809 01:46:01.567025 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87dfe669489e490ebd431e71964d7754. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:46:01.585186 10643 debug-util.cc:398] Leaking SignalData structure 0x7b080009aec0 after lost signal to thread 10535
W20260809 01:46:01.586112 10643 debug-util.cc:398] Leaking SignalData structure 0x7b08000bce40 after lost signal to thread 10651
I20260809 01:46:01.591006 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 60176118d61548ff96b001ab0dae7d21 (table default.Table_3dd6aaf3d0d545529f081db48de38cc2 [id=b90c58e7b4a34ea2a22ec6e5aa4be490]). Expected version 1 got 0
I20260809 01:46:01.592110 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 9465f8bb39d64e1cab8646282cb65c75 (table default.Table_72e975bfbe594052a24baee87785c71a [id=04d4b772004b44e781c255cf271fab34]). Expected version 5 got 0
I20260809 01:46:01.592828 10595 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_bbf8548d28c748e19d427c4f7f79622f [id=7e63b2e8084c4dc9b53d011ba1d822e6] alter complete (version 1)
I20260809 01:46:01.589147 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_81ae36e5d84645d9a61cf6a34b000deb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:01.595832 10595 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.67:40463 (request call id 62) took 1854 ms (client timeout 1000 ms). Trace:
W20260809 01:46:01.599167 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_81ae36e5d84645d9a61cf6a34b000deb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:01.602604 12931 ts_manager.cc:194] Registered new tserver with Master: 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:01.605046 11346 tablet_service.cc:1558] Processing DeleteTablet for tablet 91941616dfdd4df3aaa7c8460876d273 with delete_type TABLET_DATA_DELETED (replaced by tablet c11b0394533a485294ace9ee5e99bb19) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:01.607342 10586 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:37956 (request call id 464) took 1625 ms. Trace:
W20260809 01:46:01.607813 10599 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.66:57751 (request call id 67) took 1718 ms (client timeout 1000 ms). Trace:
W20260809 01:46:01.608129 10599 rpcz_store.cc:269] 0809 01:45:59.889483 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:59.889600 (+ 117us) service_pool.cc:224] Handling call
0809 01:46:01.607741 (+1718141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10198,"reported_tablets":10}
I20260809 01:46:01.609637 10599 master_service.cc:960] ReplaceTablet: received request to replace tablet e0fdf4f4b1d44979936b37886bc60836 from {username='slave'} at 127.0.0.1:37956
W20260809 01:46:01.596118 10595 rpcz_store.cc:269] 0809 01:45:59.741086 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:59.741287 (+ 201us) service_pool.cc:224] Handling call
0809 01:46:01.595774 (+1854487us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:59.947181 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:59.947537 (+ 356us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:59.947558 (+ 21us) write_op.cc:435] Acquired schema lock
0809 01:45:59.947571 (+ 13us) tablet.cc:662] Decoding operations
0809 01:45:59.947795 (+ 224us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:59.947825 (+ 30us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:59.947845 (+ 20us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:59.947968 (+ 123us) tablet.cc:701] Row locks acquired
0809 01:45:59.947981 (+ 13us) write_op.cc:260] PREPARE: finished
0809 01:45:59.948074 (+ 93us) write_op.cc:270] Start()
0809 01:45:59.948144 (+ 70us) write_op.cc:276] Timestamp: P: 1786239959948031 usec, L: 0
0809 01:45:59.948158 (+ 14us) op_driver.cc:348] REPLICATION: starting
0809 01:45:59.948570 (+ 412us) log.cc:844] Serialized 356 byte log entry
0809 01:45:59.976724 (+ 28154us) op_driver.cc:464] REPLICATION: finished
0809 01:45:59.978656 (+ 1932us) write_op.cc:301] APPLY: starting
0809 01:45:59.978709 (+ 53us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:59.978795 (+ 86us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:59.978869 (+ 74us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:59.979009 (+ 140us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:59.979126 (+ 117us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:59.979165 (+ 39us) write_op.cc:312] APPLY: finished
0809 01:45:59.979303 (+ 138us) log.cc:844] Serialized 23 byte log entry
0809 01:45:59.979414 (+ 111us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:59.979467 (+ 53us) write_op.cc:454] Released schema lock
0809 01:45:59.979498 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:59.979701 (+ 203us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:59.979729 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":661,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":1807,"apply.run_cpu_time_us":1015,"apply.run_wall_time_us":1016,"num_ops":1,"prepare.queue_time_us":80854,"prepare.run_cpu_time_us":1542,"prepare.run_wall_time_us":1541,"replication_time_us":28478}]]}
I20260809 01:46:01.612274 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_a7e3291583454a0a83b95fa7312eb2b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:01.607623 10586 rpcz_store.cc:276] 0809 01:45:59.981677 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:59.981800 (+ 123us) service_pool.cc:224] Handling call
0809 01:45:59.982717 (+ 917us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:45:59.983527 (+ 810us) catalog_manager.cc:3751] Apply alter schema
0809 01:45:59.984100 (+ 573us) catalog_manager.cc:3783] Acquired catalog manager lock
0809 01:45:59.984277 (+ 177us) catalog_manager.cc:3941] Updating metadata on disk
0809 01:46:00.031195 (+ 46918us) catalog_manager.cc:3972] Committing alterations to in-memory state
0809 01:46:01.607285 (+1576090us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:59.988742 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:59.989107 (+ 365us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:59.989131 (+ 24us) write_op.cc:435] Acquired schema lock
0809 01:45:59.989145 (+ 14us) tablet.cc:662] Decoding operations
0809 01:45:59.989378 (+ 233us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:59.989424 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:59.989445 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:59.989614 (+ 169us) tablet.cc:701] Row locks acquired
0809 01:45:59.989630 (+ 16us) write_op.cc:260] PREPARE: finished
0809 01:45:59.989743 (+ 113us) write_op.cc:270] Start()
0809 01:45:59.989813 (+ 70us) write_op.cc:276] Timestamp: P: 1786239959989695 usec, L: 0
0809 01:45:59.989828 (+ 15us) op_driver.cc:348] REPLICATION: starting
0809 01:45:59.994029 (+ 4201us) log.cc:844] Serialized 405 byte log entry
0809 01:46:00.029649 (+ 35620us) op_driver.cc:464] REPLICATION: finished
0809 01:46:00.029921 (+ 272us) write_op.cc:301] APPLY: starting
0809 01:46:00.029973 (+ 52us) tablet.cc:1367] starting BulkCheckPresence
0809 01:46:00.030054 (+ 81us) tablet.cc:1370] finished BulkCheckPresence
0809 01:46:00.030151 (+ 97us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:46:00.030321 (+ 170us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:46:00.030471 (+ 150us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:46:00.030525 (+ 54us) write_op.cc:312] APPLY: finished
0809 01:46:00.030700 (+ 175us) log.cc:844] Serialized 23 byte log entry
0809 01:46:00.030855 (+ 155us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:46:00.030940 (+ 85us) write_op.cc:454] Released schema lock
0809 01:46:00.030974 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0809 01:46:00.031615 (+ 641us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:46:00.031645 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13521,"child_traces":[["op",{"apply.queue_time_us":139,"apply.run_cpu_time_us":1293,"apply.run_wall_time_us":1651,"num_ops":1,"prepare.queue_time_us":2848,"prepare.run_cpu_time_us":1765,"prepare.run_wall_time_us":5466,"replication_time_us":39719}]]}
W20260809 01:46:01.613855 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a7e3291583454a0a83b95fa7312eb2b7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:01.614363 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_52926e76c3ad4bbcbc425c32d4473a3c" } new_table_name: "default.Table_04b9c11e640741448347b04d2c368e23" modify_external_catalogs: true
I20260809 01:46:01.575434 13237 consensus_queue.cc:1048] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:46:01.628367 10598 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 68f022e6a57a4ec0a5a8c33159e97a8e
I20260809 01:46:01.630774 11346 tablet_service.cc:1558] Processing DeleteTablet for tablet 68f022e6a57a4ec0a5a8c33159e97a8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:01 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:01.631976 10598 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:37956 (request call id 471) took 1337 ms. Trace:
I20260809 01:46:01.632369 10598 rpcz_store.cc:276] 0809 01:46:00.294298 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:00.301262 (+ 6964us) service_pool.cc:224] Handling call
0809 01:46:00.302593 (+ 1331us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0809 01:46:01.601072 (+1298479us) catalog_manager.cc:2841] Modifying in-memory table state
0809 01:46:01.601192 (+ 120us) catalog_manager.cc:2850] Locking tablets
0809 01:46:01.601876 (+ 684us) catalog_manager.cc:2864] Removing table and tablets from system table
0809 01:46:01.627912 (+ 26036us) catalog_manager.cc:2882] Removing table from by-name map
0809 01:46:01.628105 (+ 193us) catalog_manager.cc:2898] Committing in-memory state
0809 01:46:01.628287 (+ 182us) catalog_manager.cc:2902] Aborting table tasks
0809 01:46:01.631919 (+ 3632us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:46:01.603791 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:46:01.604263 (+ 472us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:46:01.604288 (+ 25us) write_op.cc:435] Acquired schema lock
0809 01:46:01.604299 (+ 11us) tablet.cc:662] Decoding operations
0809 01:46:01.604665 (+ 366us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:46:01.604722 (+ 57us) write_op.cc:641] Partition lock acquired for write op
0809 01:46:01.604751 (+ 29us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:46:01.605197 (+ 446us) tablet.cc:701] Row locks acquired
0809 01:46:01.605209 (+ 12us) write_op.cc:260] PREPARE: finished
0809 01:46:01.605322 (+ 113us) write_op.cc:270] Start()
0809 01:46:01.605420 (+ 98us) write_op.cc:276] Timestamp: P: 1786239961605272 usec, L: 0
0809 01:46:01.605435 (+ 15us) op_driver.cc:348] REPLICATION: starting
0809 01:46:01.606074 (+ 639us) log.cc:844] Serialized 765 byte log entry
0809 01:46:01.624653 (+ 18579us) op_driver.cc:464] REPLICATION: finished
0809 01:46:01.625048 (+ 395us) write_op.cc:301] APPLY: starting
0809 01:46:01.625120 (+ 72us) tablet.cc:1367] starting BulkCheckPresence
0809 01:46:01.625276 (+ 156us) tablet.cc:1370] finished BulkCheckPresence
0809 01:46:01.625432 (+ 156us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:46:01.625748 (+ 316us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:46:01.625947 (+ 199us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:46:01.625998 (+ 51us) write_op.cc:312] APPLY: finished
0809 01:46:01.626232 (+ 234us) log.cc:844] Serialized 29 byte log entry
0809 01:46:01.626433 (+ 201us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:46:01.626548 (+ 115us) write_op.cc:454] Released schema lock
0809 01:46:01.626586 (+ 38us) write_op.cc:341] FINISH: Updating metrics
0809 01:46:01.626932 (+ 346us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:46:01.626964 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":231,"apply.run_cpu_time_us":1804,"apply.run_wall_time_us":1804,"num_ops":2,"prepare.queue_time_us":133,"prepare.run_cpu_time_us":2610,"prepare.run_wall_time_us":4860,"raft.queue_time_us":103,"raft.run_cpu_time_us":688,"raft.run_wall_time_us":688,"replication_time_us":19100}]]}
I20260809 01:46:01.637181 10598 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_1391d3ee4c884a43b3fa55ce4cf9d7d4" } modify_external_catalogs: true
I20260809 01:46:01.637784 10598 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_1391d3ee4c884a43b3fa55ce4cf9d7d4" } modify_external_catalogs: true
I20260809 01:46:01.593400 13161 rpcz_store.cc:276] 0809 01:45:59.128262 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:59.128674 (+ 412us) service_pool.cc:224] Handling call
0809 01:46:00.922322 (+1793648us) ts_tablet_manager.cc:1151] Deleting tablet b9493892c1ff48e7a0558ff0fcb7a0f3
0809 01:46:00.959263 (+ 36941us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:01.220835 (+261572us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:01.458077 (+237242us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:01.566368 (+108291us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1745841}
I20260809 01:46:01.644418 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet b9493892c1ff48e7a0558ff0fcb7a0f3 (table default.Table_a1e1833d52934d0b90c8e6f5046d03df [id=59cbeb450bbc4fb99011118dbfe5809a]) successfully deleted
I20260809 01:46:01.659770 11334 tablet_service.cc:1558] Processing DeleteTablet for tablet a38bd344108649ed8e1687f82154000b with delete_type TABLET_DATA_DELETED (replaced by tablet 878b7a9d92df4824902b2b15295f65df) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:01.659788 11346 tablet_service.cc:1558] Processing DeleteTablet for tablet b9493892c1ff48e7a0558ff0fcb7a0f3 with delete_type TABLET_DATA_DELETED (replaced by tablet 83dd3b3eb484407bbef0e54581b759dd) from {username='slave'} at 127.0.0.1:53476
W20260809 01:46:01.660935 10597 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.67:40463 (request call id 63) took 814 ms (client timeout 1000 ms). Trace:
I20260809 01:46:01.664352 11346 tablet_service.cc:1558] Processing DeleteTablet for tablet cd09bea5263344919ce26593fe958547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:53476
W20260809 01:46:01.661258 10597 rpcz_store.cc:269] 0809 01:46:00.846299 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:00.856783 (+ 10484us) service_pool.cc:224] Handling call
0809 01:46:01.660872 (+804089us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:46:01.288212 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:46:01.288568 (+ 356us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:46:01.288591 (+ 23us) write_op.cc:435] Acquired schema lock
0809 01:46:01.288603 (+ 12us) tablet.cc:662] Decoding operations
0809 01:46:01.288865 (+ 262us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:46:01.288900 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0809 01:46:01.288921 (+ 21us) tablet.cc:685] Acquiring locks for 2 operations
0809 01:46:01.289213 (+ 292us) tablet.cc:701] Row locks acquired
0809 01:46:01.289225 (+ 12us) write_op.cc:260] PREPARE: finished
0809 01:46:01.289328 (+ 103us) write_op.cc:270] Start()
0809 01:46:01.289397 (+ 69us) write_op.cc:276] Timestamp: P: 1786239961289271 usec, L: 0
0809 01:46:01.289412 (+ 15us) op_driver.cc:348] REPLICATION: starting
0809 01:46:01.289962 (+ 550us) log.cc:844] Serialized 901 byte log entry
0809 01:46:01.311825 (+ 21863us) op_driver.cc:464] REPLICATION: finished
0809 01:46:01.318390 (+ 6565us) write_op.cc:301] APPLY: starting
0809 01:46:01.318466 (+ 76us) tablet.cc:1367] starting BulkCheckPresence
0809 01:46:01.318614 (+ 148us) tablet.cc:1370] finished BulkCheckPresence
0809 01:46:01.318714 (+ 100us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:46:01.319034 (+ 320us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:46:01.319205 (+ 171us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 01:46:01.319261 (+ 56us) write_op.cc:312] APPLY: finished
0809 01:46:01.319521 (+ 260us) log.cc:844] Serialized 29 byte log entry
0809 01:46:01.329585 (+ 10064us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:46:01.329754 (+ 169us) write_op.cc:454] Released schema lock
0809 01:46:01.329824 (+ 70us) write_op.cc:341] FINISH: Updating metrics
0809 01:46:01.330182 (+ 358us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:46:01.330224 (+ 42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6906,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":6367,"apply.run_cpu_time_us":1841,"apply.run_wall_time_us":11747,"num_ops":2,"prepare.queue_time_us":88,"prepare.run_cpu_time_us":2034,"prepare.run_wall_time_us":2041,"raft.queue_time_us":295,"raft.run_cpu_time_us":806,"raft.run_wall_time_us":806,"replication_time_us":22329}]]}
W20260809 01:46:01.666142 10589 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 60) took 1830 ms (client timeout 1000 ms). Trace:
W20260809 01:46:01.666472 10589 rpcz_store.cc:269] 0809 01:45:59.835661 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:59.835797 (+ 136us) service_pool.cc:224] Handling call
0809 01:46:01.666083 (+1830286us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17041,"reported_tablets":19}
W20260809 01:46:01.670383 10588 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.8.65:36985 (request call id 59) took 3228 ms (client timeout 1000 ms). Trace:
W20260809 01:46:01.670706 10588 rpcz_store.cc:269] 0809 01:45:58.442196 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:58.489063 (+ 46867us) service_pool.cc:224] Handling call
0809 01:45:58.794089 (+305026us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b5400030554. stack: 00007fa2bf68ccf3 00007fa2bf68cbf0 00007fa2bf23a574 0000000000516de4 0000000000516d79 00007fa2c678e470 00007fa2c682d31a 00007fa2c859705c 00007fa2c8693c7e 00007fa2c6241fb4 00007fa2c6241f42 00007fa2c6241e71 00007fa2c6241dec 00007fa2c62410a2 00007fa2c44ef07c 00007fa2c44ee4b6
0809 01:46:01.670322 (+2876233us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:45:59.804961 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 01:45:59.805311 (+ 350us) write_op.cc:432] Acquiring schema lock in shared mode
0809 01:45:59.805335 (+ 24us) write_op.cc:435] Acquired schema lock
0809 01:45:59.805347 (+ 12us) tablet.cc:662] Decoding operations
0809 01:45:59.805587 (+ 240us) write_op.cc:620] Acquiring the partition lock for write op
0809 01:45:59.805620 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0809 01:45:59.805640 (+ 20us) tablet.cc:685] Acquiring locks for 1 operations
0809 01:45:59.805799 (+ 159us) tablet.cc:701] Row locks acquired
0809 01:45:59.805813 (+ 14us) write_op.cc:260] PREPARE: finished
0809 01:45:59.805917 (+ 104us) write_op.cc:270] Start()
0809 01:45:59.805989 (+ 72us) write_op.cc:276] Timestamp: P: 1786239959805834 usec, L: 0
0809 01:45:59.806003 (+ 14us) op_driver.cc:348] REPLICATION: starting
0809 01:45:59.806472 (+ 469us) log.cc:844] Serialized 527 byte log entry
0809 01:45:59.826250 (+ 19778us) op_driver.cc:464] REPLICATION: finished
0809 01:45:59.828070 (+ 1820us) write_op.cc:301] APPLY: starting
0809 01:45:59.828114 (+ 44us) tablet.cc:1367] starting BulkCheckPresence
0809 01:45:59.828190 (+ 76us) tablet.cc:1370] finished BulkCheckPresence
0809 01:45:59.828268 (+ 78us) tablet.cc:1372] starting ApplyRowOperation cycle
0809 01:45:59.828404 (+ 136us) tablet.cc:1383] finished ApplyRowOperation cycle
0809 01:45:59.828519 (+ 115us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 01:45:59.828560 (+ 41us) write_op.cc:312] APPLY: finished
0809 01:45:59.828712 (+ 152us) log.cc:844] Serialized 23 byte log entry
0809 01:45:59.828816 (+ 104us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:59.828862 (+ 46us) write_op.cc:454] Released schema lock
0809 01:45:59.828901 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0809 01:45:59.829094 (+ 193us) write_op.cc:489] Releasing partition, row and schema locks
0809 01:45:59.829124 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16524,"reported_tablets":14,"spinlock_wait_cycles":618812800,"child_traces":[["op",{"apply.queue_time_us":1698,"apply.run_cpu_time_us":992,"apply.run_wall_time_us":993,"num_ops":1,"prepare.queue_time_us":32919,"prepare.run_cpu_time_us":1701,"prepare.run_wall_time_us":3176,"replication_time_us":20158}]]}
I20260809 01:46:01.484164 10847 tablet_service.cc:1511] Processing CreateTablet for tablet 438aecb7e5e544b393edbbe5df7966a9 (DEFAULT_TABLE table=default.Table_985df37273284bb7be1a28211209ecfa [id=386aa8599e21406397d1b975dc9b7fd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.690878 10847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 438aecb7e5e544b393edbbe5df7966a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.492024 10871 tablet_service.cc:1511] Processing CreateTablet for tablet fcb8b3e19ceb4988aad0ca6b4f6b4514 (DEFAULT_TABLE table=default.Table_190f8292dbc54632996f644b10d5c284 [id=a1015ae30d514882b85e1f704146ebf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.694159 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcb8b3e19ceb4988aad0ca6b4f6b4514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.519209 10905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7a511512184f53921cea081f82aa37" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:46:01.697782 10905 raft_consensus.cc:3060] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:01.541754 10855 tablet_service.cc:1511] Processing CreateTablet for tablet 87dfe669489e490ebd431e71964d7754 (DEFAULT_TABLE table=default.Table_a5ecac2f48c1491fb074981ce05ef950 [id=a8acdfdf9aff4072a75d0dc017880b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.702280 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87dfe669489e490ebd431e71964d7754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.649267 13083 raft_consensus.cc:493] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:01.703228 13083 raft_consensus.cc:515] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.705217 13083 leader_election.cc:290] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:01.657999 10864 tablet_service.cc:1558] Processing DeleteTablet for tablet 14b1d03085eb4ab69789fed6fab882ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:59 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:01.659610 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet b3b02bd8c93b43bf90ff0192a9541aae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:01.632951 10858 tablet_service.cc:1558] Processing DeleteTablet for tablet 68f022e6a57a4ec0a5a8c33159e97a8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:01 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:01.706558 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc9c8e0f97e41b48a919aa579ad85cd" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:01.707829 10805 leader_election.cc:343] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:01.714428 13083 raft_consensus.cc:493] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:01.714911 13083 raft_consensus.cc:515] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.716555 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 385) took 2279 ms. Trace:
I20260809 01:46:01.716894 13161 rpcz_store.cc:276] 0809 01:45:59.439190 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:59.502342 (+ 63152us) service_pool.cc:224] Handling call
0809 01:46:01.716199 (+2213857us) ts_tablet_manager.cc:1151] Deleting tablet f4985b1715f64adaae432d46dee4ea38
0809 01:46:01.716246 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:01.716495 (+ 249us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2213353}
I20260809 01:46:01.718482 13083 leader_election.cc:290] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:01.732978 10905 raft_consensus.cc:2468] T 7c7a511512184f53921cea081f82aa37 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:46:01.719161 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058dfc9b37f04fd8a72981b4290390b4" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:01.743803 10805 leader_election.cc:343] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:01.743932 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 386) took 2227 ms. Trace:
I20260809 01:46:01.744371 13161 rpcz_store.cc:276] 0809 01:45:59.516563 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:59.516683 (+ 120us) service_pool.cc:224] Handling call
0809 01:46:01.743560 (+2226877us) ts_tablet_manager.cc:1151] Deleting tablet d18d1f8343ed4c33a70fccbecc3012a6
0809 01:46:01.743608 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:01.743876 (+ 268us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2226387}
W20260809 01:46:01.744863 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet f4985b1715f64adaae432d46dee4ea38 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f4985b1715f64adaae432d46dee4ea38 already in progress: creating tablet
I20260809 01:46:01.745975 11111 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fffa48ccf58476da79dccd9c008b67c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:53 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:01.719900 11554 tablet_bootstrap.cc:492] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:46:01.746834 11551 tablet_bootstrap.cc:492] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:46:01.746905 11063 leader_election.cc:304] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:46:01.748813 11120 tablet_service.cc:1558] Processing DeleteTablet for tablet 977694496e014ef29f5d7d513e5219c0 with delete_type TABLET_DATA_DELETED (replaced by tablet b095338eb59e4161aaaf3afb762cd296) from {username='slave'} at 127.0.0.1:56566
W20260809 01:46:01.724340 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 153305us
I20260809 01:46:01.726358 13237 consensus_queue.cc:1048] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:46:01.755262 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet 68f022e6a57a4ec0a5a8c33159e97a8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:01 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:01.754827 13318 tablet.cc:1724] T cfb9afd3be194830ba99f5e20fdaca0a P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260809 01:46:01.756575 10576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.126:42157, user_credentials={real_user=slave}} blocked reactor thread for 52859.9us
I20260809 01:46:01.639024 13169 leader_election.cc:290] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:01.762971 11063 leader_election.cc:304] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:46:01.767791 11120 tablet_service.cc:1558] Processing DeleteTablet for tablet 761370ec8b164fd184a2e091030f610b with delete_type TABLET_DATA_DELETED (replaced by tablet 21e28316ad9442e2bcffba726d13cfe9) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:01.776301 10858 tablet_service.cc:1511] Processing CreateTablet for tablet edc25b46fdaf41b3b7ebab1237266b98 (DEFAULT_TABLE table=default.Table_2782952361d24cf9aa19e0667a8a67ee [id=87628394e909476a9dc338f1776a495c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.777237 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc25b46fdaf41b3b7ebab1237266b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.779816 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058dfc9b37f04fd8a72981b4290390b4" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:46:01.786453 10804 leader_election.cc:343] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:01.786893 10804 leader_election.cc:304] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:01.788053 13083 raft_consensus.cc:2749] T 058dfc9b37f04fd8a72981b4290390b4 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:01.796308 10591 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 3e03033672e947eab8cc143263281133 (table default.Table_109c891eea264584b9e1ceab647f32aa [id=24d2e675d969421fba4647380880f606]). Expected version 3 got 0
W20260809 01:46:01.798197 10586 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6f771c8051784da18a6de9fcf45e0125 consensus state has no leader
I20260809 01:46:01.797973 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc9c8e0f97e41b48a919aa579ad85cd" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:01.779515 13310 tablet_replica.cc:333] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:46:01.801615 13310 raft_consensus.cc:2243] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:01.805757 13169 raft_consensus.cc:2804] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:01.806392 13169 raft_consensus.cc:697] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:46:01.807545 13169 consensus_queue.cc:237] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:46:01.776841 13264 raft_consensus.cc:2804] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:01.749442 13317 tablet.cc:1724] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:46:01.813400 13310 raft_consensus.cc:2272] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:01.817113 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_f79203f002e14b909a945be94e3e4782" } new_table_name: "default.Table_b8e0c22368e34528bccc0ebc76aa7025" modify_external_catalogs: true
I20260809 01:46:01.818384 10599 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet e0fdf4f4b1d44979936b37886bc60836
W20260809 01:46:01.820192 10804 leader_election.cc:343] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:01.820662 10804 leader_election.cc:304] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:01.821580 10599 catalog_manager.cc:6807] ReplaceTablet: tablet e0fdf4f4b1d44979936b37886bc60836 deleted and replaced by tablet 65fa053480ee446685796edc48772aa5
I20260809 01:46:01.822865 13325 raft_consensus.cc:2749] T acc9c8e0f97e41b48a919aa579ad85cd P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:01.825354 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet e0fdf4f4b1d44979936b37886bc60836 with delete_type TABLET_DATA_DELETED (replaced by tablet 65fa053480ee446685796edc48772aa5) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:01.826936 10597 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_cdc79f33d04b4f6db40aa0dd540b8b4d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:01.828326 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet bb7a8b6d06064addb02979a7feedf756 with delete_type TABLET_DATA_DELETED (replaced by tablet 2126670a2f29408a9ba4b949f966bf76) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:01.828675 11120 tablet_service.cc:1558] Processing DeleteTablet for tablet 8323a7a40a1445f980c0adcf6a6d16c7 with delete_type TABLET_DATA_DELETED (replaced by tablet 012b043d7eef472eb30bdd8e3eec271f) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:01.829241 10588 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_95568eee8c064b09bcb94444d7839a77"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:01.830868 10588 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_95568eee8c064b09bcb94444d7839a77 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:01.832154 11346 tablet_service.cc:1558] Processing DeleteTablet for tablet e0fdf4f4b1d44979936b37886bc60836 with delete_type TABLET_DATA_DELETED (replaced by tablet 65fa053480ee446685796edc48772aa5) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:01.833870 11116 tablet_service.cc:1511] Processing CreateTablet for tablet edc25b46fdaf41b3b7ebab1237266b98 (DEFAULT_TABLE table=default.Table_2782952361d24cf9aa19e0667a8a67ee [id=87628394e909476a9dc338f1776a495c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.839601 10599 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_20f4c0cc067f4302a50669bc5303985a" } modify_external_catalogs: true
I20260809 01:46:01.840103 10599 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_20f4c0cc067f4302a50669bc5303985a" } modify_external_catalogs: true
I20260809 01:46:01.841059 11120 tablet_service.cc:1558] Processing DeleteTablet for tablet cd09bea5263344919ce26593fe958547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:42 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:01.841838 13323 consensus_queue.cc:1048] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:46:01.843550 13310 ts_tablet_manager.cc:1916] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:01.852696 13237 consensus_queue.cc:1048] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:46:01.846374 11551 tablet_bootstrap.cc:654] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:01.849684 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc25b46fdaf41b3b7ebab1237266b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.862759 13309 raft_consensus.cc:493] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:01.863234 13309 raft_consensus.cc:515] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:46:01.865803 10597 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cdc79f33d04b4f6db40aa0dd540b8b4d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:46:01.881956 10581 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet b5bd08670e2e40e0bb250d5e054a4116 (table default.Table_81380e3f093c48cb8811640969271318 [id=13f1a063688946178a8a02abac0401b2], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b5bd08670e2e40e0bb250d5e054a4116 consensus state has no leader
I20260809 01:46:01.888406 11346 tablet_service.cc:1511] Processing CreateTablet for tablet b96660c00211448486fffee6305322b1 (DEFAULT_TABLE table=default.Table_eb837ff4a9784eed8fca586c6c6b53ec [id=dc24b0aa7d63414e91474b2438be23e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:01.891921 13083 raft_consensus.cc:493] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:01.892688 13083 raft_consensus.cc:515] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.897460 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cc1253dee24f5d858d521ddbbfb620" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:01.899178 10805 leader_election.cc:343] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:01.745687 13161 tablet_replica.cc:333] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:46:01.900205 13161 raft_consensus.cc:2243] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:01.900447 13083 leader_election.cc:290] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:01.900715 13161 raft_consensus.cc:2272] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:01.902768 13161 ts_tablet_manager.cc:1916] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:01.910534 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b96660c00211448486fffee6305322b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:01.914521 10592 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 3e03033672e947eab8cc143263281133 (table default.Table_109c891eea264584b9e1ceab647f32aa [id=24d2e675d969421fba4647380880f606]). Expected version 3 got 0
I20260809 01:46:01.914520 13309 leader_election.cc:290] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:46:01.920670 11289 consensus_peers.cc:597] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 761370ec8b164fd184a2e091030f610b. This is attempt 1: this message will repeat every 5th retry.
W20260809 01:46:01.922097 11289 consensus_peers.cc:597] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cd09bea5263344919ce26593fe958547. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:46:01.955535 13264 raft_consensus.cc:697] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:46:01.956679 13264 consensus_queue.cc:237] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.975306 13334 raft_consensus.cc:493] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:01.975906 11119 tablet_service.cc:1558] Processing DeleteTablet for tablet e0fdf4f4b1d44979936b37886bc60836 with delete_type TABLET_DATA_DELETED (replaced by tablet 65fa053480ee446685796edc48772aa5) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:01.976173 13334 raft_consensus.cc:515] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:01.981910 13334 leader_election.cc:290] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:01.993862 11554 tablet_bootstrap.cc:654] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:01.999799 11553 tablet_bootstrap.cc:492] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:46:02.000190 11553 ts_tablet_manager.cc:1403] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.768s user 0.015s sys 0.015s
I20260809 01:46:02.020965 10591 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_33cbb047c61649c3943ed7add66fbe48" } new_table_name: "default.Table_53d6ec64b416401fa38becd6c58addfe" modify_external_catalogs: true
I20260809 01:46:02.021176 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_ff36f56ffe4640418291a7bc746caa81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:02.022864 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff36f56ffe4640418291a7bc746caa81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:02.025406 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_921cdb8af3a2476cb5b87c0bf56902f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:02.026901 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_921cdb8af3a2476cb5b87c0bf56902f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:46:02.005484 11062 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s user 0.034s sys 0.059s
W20260809 01:46:02.033996 11062 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s user 0.034s sys 0.060s
W20260809 01:46:02.034456 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 593614us
W20260809 01:46:02.035178 11062 leader_election.cc:343] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.006910 11551 tablet_bootstrap.cc:492] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:46:02.036599 11551 ts_tablet_manager.cc:1403] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.290s user 0.013s sys 0.001s
W20260809 01:46:02.037230 11062 leader_election.cc:343] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.038110 10599 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet acc9c8e0f97e41b48a919aa579ad85cd
W20260809 01:46:02.038172 11062 leader_election.cc:343] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.045621 10598 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 5e9afec0618e4fa293f6dad6769ae49c
I20260809 01:46:02.050122 10861 tablet_service.cc:1511] Processing CreateTablet for tablet b96660c00211448486fffee6305322b1 (DEFAULT_TABLE table=default.Table_eb837ff4a9784eed8fca586c6c6b53ec [id=dc24b0aa7d63414e91474b2438be23e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.050899 10869 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e9afec0618e4fa293f6dad6769ae49c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:01 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:02.051391 10858 tablet_service.cc:1511] Processing CreateTablet for tablet b5bd08670e2e40e0bb250d5e054a4116 (DEFAULT_TABLE table=default.Table_81380e3f093c48cb8811640969271318 [id=13f1a063688946178a8a02abac0401b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.051765 10871 tablet_service.cc:1558] Processing DeleteTablet for tablet acc9c8e0f97e41b48a919aa579ad85cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:01 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:02.052563 10855 tablet_service.cc:1511] Processing CreateTablet for tablet 1f9d2812ea2d4404a5aa61f517b8762c (DEFAULT_TABLE table=default.Table_1118918607d4448c8ec1a89b98973711 [id=18261916ad7344e79242f3d6b4749242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.057622 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325e3d19b83545a197c2e09fc9643b5e" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:46:02.058846 10589 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 325e3d19b83545a197c2e09fc9643b5e (table default.Table_66160146b5684111a0b61024efd70dfb [id=ccd0ea07da604f859047f2eca85fc566]). Expected version 3 got 0
I20260809 01:46:02.059851 10589 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 3e03033672e947eab8cc143263281133 (table default.Table_109c891eea264584b9e1ceab647f32aa [id=24d2e675d969421fba4647380880f606]). Expected version 3 got 0
I20260809 01:46:02.060810 11360 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e9afec0618e4fa293f6dad6769ae49c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:01 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:02.061365 11334 tablet_service.cc:1558] Processing DeleteTablet for tablet acc9c8e0f97e41b48a919aa579ad85cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:01 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:02.061465 11347 tablet_service.cc:1558] Processing DeleteTablet for tablet 761370ec8b164fd184a2e091030f610b with delete_type TABLET_DATA_DELETED (replaced by tablet 21e28316ad9442e2bcffba726d13cfe9) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:02.062018 11366 tablet_service.cc:1558] Processing DeleteTablet for tablet 8323a7a40a1445f980c0adcf6a6d16c7 with delete_type TABLET_DATA_DELETED (replaced by tablet 012b043d7eef472eb30bdd8e3eec271f) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:02.064853 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 1f9d2812ea2d4404a5aa61f517b8762c (DEFAULT_TABLE table=default.Table_1118918607d4448c8ec1a89b98973711 [id=18261916ad7344e79242f3d6b4749242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.066075 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f9d2812ea2d4404a5aa61f517b8762c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.070722 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d5a709cd86476594613e788a325094" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:46:02.071007 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bc7dc3dfc2f4c1286ca24b8e3b20719" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:46:02.029300 13237 raft_consensus.cc:493] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:02.071549 11395 raft_consensus.cc:2468] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:46:02.071763 13237 raft_consensus.cc:515] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.074416 13237 leader_election.cc:290] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:02.076788 11351 tablet_service.cc:1511] Processing CreateTablet for tablet b5bd08670e2e40e0bb250d5e054a4116 (DEFAULT_TABLE table=default.Table_81380e3f093c48cb8811640969271318 [id=13f1a063688946178a8a02abac0401b2]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:46:02.077277 11289 leader_election.cc:343] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 8323a7a40a1445f980c0adcf6a6d16c7
I20260809 01:46:02.078016 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5bd08670e2e40e0bb250d5e054a4116. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:46:02.065603 10576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.126:42157, user_credentials={real_user=slave}} blocked reactor thread for 45997.9us
W20260809 01:46:02.084946 11349 consensus_meta.cc:287] T edc25b46fdaf41b3b7ebab1237266b98 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent flushing consensus metadata: real 0.525s user 0.009s sys 0.001s
W20260809 01:46:02.018774 13349 log.cc:927] Time spent T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7: Append to log took a long time: real 0.051s user 0.000s sys 0.001s
I20260809 01:46:02.108392 13083 raft_consensus.cc:493] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:02.108856 13083 raft_consensus.cc:515] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.112455 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a238ad3d46054daa9435e9076ba8311d" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:02.113783 10805 leader_election.cc:343] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.110929 13083 leader_election.cc:290] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:46:02.107890 11062 leader_election.cc:343] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:46:02.124070 11062 leader_election.cc:343] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.124665 11062 leader_election.cc:304] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
W20260809 01:46:02.142935 10586 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 712a7c26f2304697b6fba23767ede2a0 consensus state has no leader
I20260809 01:46:02.144999 11120 tablet_service.cc:1558] Processing DeleteTablet for tablet acc9c8e0f97e41b48a919aa579ad85cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:01 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:02.004989 11553 raft_consensus.cc:359] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.149972 11553 raft_consensus.cc:385] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:02.150321 11553 raft_consensus.cc:740] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
I20260809 01:46:02.151140 11553 consensus_queue.cc:260] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.160271 11111 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e9afec0618e4fa293f6dad6769ae49c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:01 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:02.160141 11120 tablet_service.cc:1511] Processing CreateTablet for tablet 1f9d2812ea2d4404a5aa61f517b8762c (DEFAULT_TABLE table=default.Table_1118918607d4448c8ec1a89b98973711 [id=18261916ad7344e79242f3d6b4749242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.165616 13308 ts_tablet_manager.cc:1929] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:46:02.165805 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8323a7a40a1445f980c0adcf6a6d16c7" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:02.166263 13308 log.cc:1199] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/31874b7e70a847a4885bd09da40a6902
I20260809 01:46:02.166700 11155 raft_consensus.cc:2468] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:46:02.167361 13308 ts_tablet_manager.cc:1950] T 31874b7e70a847a4885bd09da40a6902 P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:46:02.170864 11119 tablet_service.cc:1511] Processing CreateTablet for tablet b96660c00211448486fffee6305322b1 (DEFAULT_TABLE table=default.Table_eb837ff4a9784eed8fca586c6c6b53ec [id=dc24b0aa7d63414e91474b2438be23e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.173126 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b96660c00211448486fffee6305322b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.181010 11116 tablet_service.cc:1511] Processing CreateTablet for tablet b5bd08670e2e40e0bb250d5e054a4116 (DEFAULT_TABLE table=default.Table_81380e3f093c48cb8811640969271318 [id=13f1a063688946178a8a02abac0401b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.183008 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5bd08670e2e40e0bb250d5e054a4116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.185937 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f9d2812ea2d4404a5aa61f517b8762c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.190675 11283 leader_election.cc:304] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:02.192847 13319 raft_consensus.cc:2804] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:02.193288 13319 raft_consensus.cc:493] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:02.193753 13319 raft_consensus.cc:3060] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:02.200672 11120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f9d2812ea2d4404a5aa61f517b8762c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.222465 11126 tablet_service.cc:1558] Processing DeleteTablet for tablet 50f45c02a6994c5aaaf9c0a499e15fed with delete_type TABLET_DATA_DELETED (replaced by tablet ebc209c696ac4c669961f5ed8392af40) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:02.224009 11554 tablet_bootstrap.cc:492] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:02.224467 11554 ts_tablet_manager.cc:1403] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.505s user 0.008s sys 0.004s
I20260809 01:46:02.229887 11553 ts_tablet_manager.cc:1434] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.227s user 0.006s sys 0.001s
I20260809 01:46:02.243927 10586 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_51ec411bde9648a5bce1df1b21c0cf92" } modify_external_catalogs: true
I20260809 01:46:02.243639 13319 raft_consensus.cc:515] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.244534 10586 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_51ec411bde9648a5bce1df1b21c0cf92" } modify_external_catalogs: true
I20260809 01:46:02.234815 10871 tablet_service.cc:1511] Processing CreateTablet for tablet 8b4bbc6d4c594a76a3729d27926ae479 (DEFAULT_TABLE table=default.Table_148f7d52dc6c4c329484c2fc8b366693 [id=fd2b60b639e74dd8b7438cde4934fe92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.246062 10847 tablet_service.cc:1558] Processing DeleteTablet for tablet 66cc1253dee24f5d858d521ddbbfb620 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:45 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:02.248778 11553 tablet_bootstrap.cc:492] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:46:02.251194 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 31874b7e70a847a4885bd09da40a6902 (table default.Table_53624396796f4684acbc3787e6e05f6e [id=18e87d9e1fa3442aaa489e3abcd4640e]) successfully deleted
I20260809 01:46:02.251844 10588 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_45e8dad480984c1ca35198bd7cd125c3" } modify_external_catalogs: true
I20260809 01:46:02.252300 10588 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_45e8dad480984c1ca35198bd7cd125c3" } modify_external_catalogs: true
I20260809 01:46:02.263172 13308 tablet_replica.cc:333] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:46:02.266302 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5bd08670e2e40e0bb250d5e054a4116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.268098 13308 raft_consensus.cc:2243] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:02.271124 13308 raft_consensus.cc:2272] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260809 01:46:02.272293 11289 leader_election.cc:343] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 8323a7a40a1445f980c0adcf6a6d16c7
I20260809 01:46:02.282323 13319 leader_election.cc:290] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:02.281198 13083 raft_consensus.cc:493] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:02.283313 11111 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 with delete_type TABLET_DATA_DELETED (replaced by tablet 6542a2b55ce94835a31c732552d87222) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:02.283193 13083 raft_consensus.cc:515] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.289223 13083 leader_election.cc:290] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:46:02.299516 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 174918us
W20260809 01:46:02.310884 10591 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e3690212d43543fc982815dcff225a51 consensus state has no leader
I20260809 01:46:02.313145 13314 raft_consensus.cc:2804] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:02.313674 13314 raft_consensus.cc:493] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:02.314093 13314 raft_consensus.cc:3060] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:02.318627 11551 raft_consensus.cc:359] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.319239 11551 raft_consensus.cc:385] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:02.319466 11551 raft_consensus.cc:740] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:46:02.320066 11551 consensus_queue.cc:260] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.323621 11111 tablet_service.cc:1511] Processing CreateTablet for tablet 8b4bbc6d4c594a76a3729d27926ae479 (DEFAULT_TABLE table=default.Table_148f7d52dc6c4c329484c2fc8b366693 [id=fd2b60b639e74dd8b7438cde4934fe92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.324749 11119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b96660c00211448486fffee6305322b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.327409 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8323a7a40a1445f980c0adcf6a6d16c7" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:46:02.328003 11156 raft_consensus.cc:3060] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:02.284132 11349 tablet_service.cc:1511] Processing CreateTablet for tablet 8b4bbc6d4c594a76a3729d27926ae479 (DEFAULT_TABLE table=default.Table_148f7d52dc6c4c329484c2fc8b366693 [id=fd2b60b639e74dd8b7438cde4934fe92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.325596 13237 raft_consensus.cc:493] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:02.358608 13237 raft_consensus.cc:515] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.363030 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa12353c25cb4da680d478ce104bc6ba" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:02.364223 13237 leader_election.cc:290] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:02.366209 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa12353c25cb4da680d478ce104bc6ba" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:46:02.366889 10908 raft_consensus.cc:2468] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:46:02.368672 11289 leader_election.cc:304] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:46:02.369984 13237 raft_consensus.cc:2804] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:02.370826 13237 raft_consensus.cc:493] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:02.371307 13237 raft_consensus.cc:3060] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:02.373783 11361 tablet_service.cc:1511] Processing CreateTablet for tablet cf42f42854494d2685241b6cb1b01658 (DEFAULT_TABLE table=default.Table_b4e6661838ac4bd7a5fb72595976f957 [id=adb3c7f73a704174be926754919e0fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.374827 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf42f42854494d2685241b6cb1b01658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.376449 11349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b4bbc6d4c594a76a3729d27926ae479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.345428 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938bfd21b294fa6a25f0d8efc1ca29a" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:02.379606 10805 leader_election.cc:343] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.380371 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b4bbc6d4c594a76a3729d27926ae479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.352548 11554 raft_consensus.cc:359] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.381669 11554 raft_consensus.cc:385] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:02.381956 11554 raft_consensus.cc:740] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:02.391686 11554 consensus_queue.cc:260] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.399565 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_c513b1a1e5f84adcbda1576873b3ba41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:02.400964 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c513b1a1e5f84adcbda1576873b3ba41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:02.402386 10596 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_67456cecac004439bf596d7f3a46d602" } new_table_name: "default.Table_bb1ef451c04e4128a908618eb2c5c1ae" modify_external_catalogs: true
I20260809 01:46:02.406418 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_b119819658d344a0887b87b4d540e8f2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:02.407879 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b119819658d344a0887b87b4d540e8f2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:02.411293 13083 raft_consensus.cc:493] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:02.411842 13083 raft_consensus.cc:515] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.415030 13083 leader_election.cc:290] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:02.424582 11554 ts_tablet_manager.cc:1434] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.200s user 0.005s sys 0.008s
I20260809 01:46:02.427333 10586 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 74d6b03590c8472db8977bee5962aa6c
I20260809 01:46:02.433036 10588 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet c9e844ceeb364842b2aa3385c67963e7
W20260809 01:46:02.438899 11063 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s user 0.023s sys 0.085s
W20260809 01:46:02.439210 11063 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s user 0.023s sys 0.085s
W20260809 01:46:02.439442 11063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 562086us
I20260809 01:46:02.442116 11551 ts_tablet_manager.cc:1434] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.405s user 0.004s sys 0.002s
W20260809 01:46:02.443828 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 140350us
I20260809 01:46:02.450595 11105 tablet_service.cc:1558] Processing DeleteTablet for tablet 74d6b03590c8472db8977bee5962aa6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:02 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:02.452360 11105 tablet_service.cc:1558] Processing DeleteTablet for tablet c9e844ceeb364842b2aa3385c67963e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:02 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:02.434190 11346 tablet_service.cc:1558] Processing DeleteTablet for tablet 74d6b03590c8472db8977bee5962aa6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:02 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:02.435783 11345 tablet_service.cc:1558] Processing DeleteTablet for tablet c9e844ceeb364842b2aa3385c67963e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:02 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:02.438706 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34616fd5895945b3aa896d5a8ee87249" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:02.453459 11283 leader_election.cc:343] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.464468 11111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b4bbc6d4c594a76a3729d27926ae479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.464709 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cc1253dee24f5d858d521ddbbfb620" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:02.476528 12956 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33104 (request call id 0) took 5388 ms. Trace:
I20260809 01:46:02.477221 11551 tablet_bootstrap.cc:492] T 67b31998b65e4264b16c0d73dbcfc73b P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:46:02.477722 11117 tablet_service.cc:1511] Processing CreateTablet for tablet cf42f42854494d2685241b6cb1b01658 (DEFAULT_TABLE table=default.Table_b4e6661838ac4bd7a5fb72595976f957 [id=adb3c7f73a704174be926754919e0fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.479023 11117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf42f42854494d2685241b6cb1b01658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.492534 11345 tablet_service.cc:1511] Processing CreateTablet for tablet c9e844ceeb364842b2aa3385c67963e7 (DEFAULT_TABLE table=default.Table_45e8dad480984c1ca35198bd7cd125c3 [id=e5ad3c6ddd1d4efd95eb8b7c1d1e5491]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.494024 11345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9e844ceeb364842b2aa3385c67963e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.508279 11554 tablet_bootstrap.cc:492] T 7c7a511512184f53921cea081f82aa37 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:46:02.508934 11120 tablet_service.cc:1511] Processing CreateTablet for tablet c9e844ceeb364842b2aa3385c67963e7 (DEFAULT_TABLE table=default.Table_45e8dad480984c1ca35198bd7cd125c3 [id=e5ad3c6ddd1d4efd95eb8b7c1d1e5491]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.451458 13169 raft_consensus.cc:493] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:46:02.516610 10596 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a238ad3d46054daa9435e9076ba8311d consensus state has no leader
I20260809 01:46:02.436522 10855 tablet_service.cc:1558] Processing DeleteTablet for tablet c9e844ceeb364842b2aa3385c67963e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:02 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:02.436722 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet 74d6b03590c8472db8977bee5962aa6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:02 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:02.428330 11553 tablet_bootstrap.cc:654] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:02.442106 13308 ts_tablet_manager.cc:1916] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:02.442893 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d5a709cd86476594613e788a325094" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:46:02.455655 10858 tablet_service.cc:1511] Processing CreateTablet for tablet cf42f42854494d2685241b6cb1b01658 (DEFAULT_TABLE table=default.Table_b4e6661838ac4bd7a5fb72595976f957 [id=adb3c7f73a704174be926754919e0fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.528431 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf42f42854494d2685241b6cb1b01658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.545207 11554 tablet_bootstrap.cc:654] T 7c7a511512184f53921cea081f82aa37 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
W20260809 01:46:02.463394 10805 leader_election.cc:343] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:46:02.550347 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 87030us
I20260809 01:46:02.460355 13083 raft_consensus.cc:493] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:02.554590 13083 raft_consensus.cc:515] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.555956 10908 raft_consensus.cc:2468] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:46:02.560535 13371 raft_consensus.cc:493] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:02.560988 13371 raft_consensus.cc:515] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.563815 13371 leader_election.cc:290] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
W20260809 01:46:02.513599 11154 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s user 0.013s sys 0.005s
I20260809 01:46:02.568301 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c7ec3928224059ab7f58d203224536" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:02.569630 10805 leader_election.cc:343] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.572515 10861 tablet_service.cc:1511] Processing CreateTablet for tablet c9e844ceeb364842b2aa3385c67963e7 (DEFAULT_TABLE table=default.Table_45e8dad480984c1ca35198bd7cd125c3 [id=e5ad3c6ddd1d4efd95eb8b7c1d1e5491]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.573426 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9e844ceeb364842b2aa3385c67963e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.575129 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a238ad3d46054daa9435e9076ba8311d" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:02.578374 11551 tablet_bootstrap.cc:654] T 67b31998b65e4264b16c0d73dbcfc73b P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:02.578739 10855 tablet_service.cc:1511] Processing CreateTablet for tablet 39b00ef870e94bee8a9bd247e69400a6 (DEFAULT_TABLE table=default.Table_5da7b5860d5d457c8fa8e67d7f9833d5 [id=f9eab70ead9646b8a8b3c93c7b18d9dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.579921 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39b00ef870e94bee8a9bd247e69400a6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:46:02.574301 11154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s user 0.014s sys 0.005s
I20260809 01:46:02.585923 13363 tablet.cc:1724] T cfb9afd3be194830ba99f5e20fdaca0a P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:46:02.586360 11157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938bfd21b294fa6a25f0d8efc1ca29a" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:02.592728 11156 raft_consensus.cc:2468] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:46:02.593986 11283 leader_election.cc:304] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:02.595036 13273 raft_consensus.cc:2804] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:02.595433 13273 raft_consensus.cc:697] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:46:02.596208 13273 consensus_queue.cc:237] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.600456 13303 catalog_manager.cc:1295] Loaded cluster ID: 465fa00c323a404c8f72f4f6eb53200b
I20260809 01:46:02.604719 13161 ts_tablet_manager.cc:1929] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260809 01:46:02.612699 13354 log.cc:927] Time spent T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7: Append to log took a long time: real 0.122s user 0.000s sys 0.001s
W20260809 01:46:02.613595 10576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.126:42157, user_credentials={real_user=slave}} blocked reactor thread for 69389.6us
I20260809 01:46:02.616629 13083 leader_election.cc:290] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:02.550697 13169 raft_consensus.cc:515] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.619015 13169 leader_election.cc:290] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:02.619155 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_ce7493c5349e41d890087f1336689d4a" } new_table_name: "default.Table_a3fe3ca9a3c54ee9ad390799d0022776" modify_external_catalogs: true
I20260809 01:46:02.619320 10588 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_4cbc6f1c63634d409cdd3a1b450f9694"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:02.620704 10588 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4cbc6f1c63634d409cdd3a1b450f9694 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:02.622161 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_c09873b499f949898bd861175f2c9c6b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:02.623374 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34616fd5895945b3aa896d5a8ee87249" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:46:02.623754 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c09873b499f949898bd861175f2c9c6b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:02.630430 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c7ec3928224059ab7f58d203224536" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:02.633973 11153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d5a709cd86476594613e788a325094" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:02.634661 11153 raft_consensus.cc:2468] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in term 0.
I20260809 01:46:02.639019 10595 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_bb1ef451c04e4128a908618eb2c5c1ae" } modify_external_catalogs: true
I20260809 01:46:02.639475 10595 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_bb1ef451c04e4128a908618eb2c5c1ae" } modify_external_catalogs: true
I20260809 01:46:02.639740 11120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9e844ceeb364842b2aa3385c67963e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.645650 13303 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: loading cluster ID for follower catalog manager: success
I20260809 01:46:02.616035 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d5a709cd86476594613e788a325094" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:46:02.658214 13161 log.cc:1199] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/403cb09c1786491ab0d40502038bf9c0
W20260809 01:46:02.658493 10805 leader_election.cc:343] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.658921 13161 ts_tablet_manager.cc:1950] T 403cb09c1786491ab0d40502038bf9c0 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:46:02.660769 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 387) took 2931 ms. Trace:
I20260809 01:46:02.666874 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6c244108a4ff3a2df0585e5280294" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:02.669099 11062 leader_election.cc:343] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:46:02.673563 11063 leader_election.cc:343] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 1bc7dc3dfc2f4c1286ca24b8e3b20719
W20260809 01:46:02.685433 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 43727.7us
I20260809 01:46:02.689185 11105 tablet_service.cc:1511] Processing CreateTablet for tablet 39b00ef870e94bee8a9bd247e69400a6 (DEFAULT_TABLE table=default.Table_5da7b5860d5d457c8fa8e67d7f9833d5 [id=f9eab70ead9646b8a8b3c93c7b18d9dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.691442 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39b00ef870e94bee8a9bd247e69400a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.685910 10591 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_1441945ab21646eba550ae9145deb4fd" } modify_external_catalogs: true
I20260809 01:46:02.694525 10591 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_1441945ab21646eba550ae9145deb4fd" } modify_external_catalogs: true
I20260809 01:46:02.698895 11063 leader_election.cc:304] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:46:02.701859 10804 leader_election.cc:343] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.702359 10804 leader_election.cc:304] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:02.704398 13083 raft_consensus.cc:2749] T 66cc1253dee24f5d858d521ddbbfb620 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:46:02.705565 10804 leader_election.cc:343] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.705955 10804 leader_election.cc:304] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:02.706928 13083 raft_consensus.cc:2749] T a238ad3d46054daa9435e9076ba8311d P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:02.707947 11553 tablet_bootstrap.cc:492] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
W20260809 01:46:02.707981 10804 leader_election.cc:343] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.708389 11553 ts_tablet_manager.cc:1403] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.460s user 0.014s sys 0.016s
I20260809 01:46:02.708360 10804 leader_election.cc:304] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:02.713572 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 39b00ef870e94bee8a9bd247e69400a6 (DEFAULT_TABLE table=default.Table_5da7b5860d5d457c8fa8e67d7f9833d5 [id=f9eab70ead9646b8a8b3c93c7b18d9dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.714365 10597 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_666b764e538240eabf2b6da622d0832f [id=5aa4b9c7be9447a3ba1bfda3a6e689b0] alter complete (version 2)
I20260809 01:46:02.714668 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39b00ef870e94bee8a9bd247e69400a6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:46:02.722045 10586 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cb13f4c7b07348cba67b6b43a9b154bd consensus state has no leader
I20260809 01:46:02.725900 13083 raft_consensus.cc:2749] T 3938bfd21b294fa6a25f0d8efc1ca29a P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:02.727862 11553 raft_consensus.cc:359] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.728462 11553 raft_consensus.cc:385] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:02.728693 11553 raft_consensus.cc:740] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
W20260809 01:46:02.730679 10804 leader_election.cc:343] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.730820 10588 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_66160146b5684111a0b61024efd70dfb" } new_table_name: "default.Table_b21581ad07a34958a36ccbe1a72ab54d" modify_external_catalogs: true
I20260809 01:46:02.731097 10804 leader_election.cc:304] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:02.732218 13371 raft_consensus.cc:2749] T 34616fd5895945b3aa896d5a8ee87249 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:02.638504 13237 raft_consensus.cc:515] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.639322 11554 tablet_bootstrap.cc:492] T 7c7a511512184f53921cea081f82aa37 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:02.734033 11554 ts_tablet_manager.cc:1403] T 7c7a511512184f53921cea081f82aa37 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.229s user 0.011s sys 0.004s
I20260809 01:46:02.734195 10855 tablet_service.cc:1511] Processing CreateTablet for tablet 033eaf96ef5d4c6f87c28f70bde94ff0 (DEFAULT_TABLE table=default.Table_17ac5d26975e4c1592d04b6573a6758d [id=7b4191ad61c143eb8badf6c4f983ecee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.734334 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_8261d1a2fbb241ac8e38755611662bad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:02.733374 10804 leader_election.cc:343] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.734691 13380 raft_consensus.cc:493] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 629f002048f64491afdd57f72a466e59)
I20260809 01:46:02.734977 10804 leader_election.cc:304] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:02.735096 13380 raft_consensus.cc:515] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
W20260809 01:46:02.735697 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8261d1a2fbb241ac8e38755611662bad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:02.736060 10804 leader_election.cc:304] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:02.736680 13371 raft_consensus.cc:2749] T 00c7ec3928224059ab7f58d203224536 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:02.737402 13371 raft_consensus.cc:2804] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:02.737062 11554 raft_consensus.cc:359] T 7c7a511512184f53921cea081f82aa37 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.737800 13371 raft_consensus.cc:493] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:02.738144 11554 raft_consensus.cc:385] T 7c7a511512184f53921cea081f82aa37 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:02.738250 13371 raft_consensus.cc:3060] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:02.738432 11554 raft_consensus.cc:740] T 7c7a511512184f53921cea081f82aa37 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:02.738710 11351 tablet_service.cc:1511] Processing CreateTablet for tablet 457ae94c3f9b4f4dbe8cd3101e4c578b (DEFAULT_TABLE table=default.Table_99bad24613e94b6f840f637bea4978fc [id=ddfe680283854b9484dc3e9d6e626ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.739615 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 457ae94c3f9b4f4dbe8cd3101e4c578b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.739142 11554 consensus_queue.cc:260] T 7c7a511512184f53921cea081f82aa37 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.741633 11554 ts_tablet_manager.cc:1434] T 7c7a511512184f53921cea081f82aa37 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260809 01:46:02.742520 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 453 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:46:02.742777 11554 tablet_bootstrap.cc:492] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:46:02.689743 13314 raft_consensus.cc:515] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.752391 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 033eaf96ef5d4c6f87c28f70bde94ff0 (DEFAULT_TABLE table=default.Table_17ac5d26975e4c1592d04b6573a6758d [id=7b4191ad61c143eb8badf6c4f983ecee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.753814 13237 leader_election.cc:290] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:02.758906 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 457ae94c3f9b4f4dbe8cd3101e4c578b (DEFAULT_TABLE table=default.Table_99bad24613e94b6f840f637bea4978fc [id=ddfe680283854b9484dc3e9d6e626ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.760254 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 033eaf96ef5d4c6f87c28f70bde94ff0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:46:02.762548 11127 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.639s user 0.006s sys 0.016s
W20260809 01:46:02.762921 11127 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.656s user 0.007s sys 0.016s
I20260809 01:46:02.763967 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 453 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:46:02.766069 12870 leader_election.cc:304] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bc72deb1d2402cabd817dfe3b27e88; no voters: 09a73ee8e8ec496a984efcab27715506, 629f002048f64491afdd57f72a466e59
I20260809 01:46:02.767316 10595 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet a238ad3d46054daa9435e9076ba8311d
I20260809 01:46:02.770015 10591 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 4fe6cb91ac524251bc7810dab5863ee8
I20260809 01:46:02.729327 11553 consensus_queue.cc:260] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:46:02.772634 10588 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 325e3d19b83545a197c2e09fc9643b5e consensus state has no leader
I20260809 01:46:02.779883 11127 tablet_service.cc:1558] Processing DeleteTablet for tablet a238ad3d46054daa9435e9076ba8311d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:02 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:02.661062 13161 rpcz_store.cc:276] 0809 01:45:59.728969 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:59.729105 (+ 136us) service_pool.cc:224] Handling call
0809 01:46:01.745505 (+2016400us) ts_tablet_manager.cc:1151] Deleting tablet 403cb09c1786491ab0d40502038bf9c0
0809 01:46:01.745569 (+ 64us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:02.245005 (+499436us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:02.604611 (+359606us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:02.660720 (+ 56109us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1995236}
I20260809 01:46:02.771981 11346 tablet_service.cc:1558] Processing DeleteTablet for tablet a238ad3d46054daa9435e9076ba8311d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:02 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:02.795266 11553 ts_tablet_manager.cc:1434] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.087s user 0.003s sys 0.003s
I20260809 01:46:02.799611 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 403cb09c1786491ab0d40502038bf9c0 (table default.Table_63e5e407241d4eaea93f27d8285ab33a [id=9e066a2a246442b78bfc003520f4cfbd]) successfully deleted
I20260809 01:46:02.801409 11346 tablet_service.cc:1511] Processing CreateTablet for tablet 033eaf96ef5d4c6f87c28f70bde94ff0 (DEFAULT_TABLE table=default.Table_17ac5d26975e4c1592d04b6573a6758d [id=7b4191ad61c143eb8badf6c4f983ecee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.804829 11126 tablet_service.cc:1511] Processing CreateTablet for tablet 457ae94c3f9b4f4dbe8cd3101e4c578b (DEFAULT_TABLE table=default.Table_99bad24613e94b6f840f637bea4978fc [id=ddfe680283854b9484dc3e9d6e626ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.762197 13380 leader_election.cc:290] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:46:02.805908 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 457ae94c3f9b4f4dbe8cd3101e4c578b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.805881 13380 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 01:46:02.775888 10847 tablet_service.cc:1558] Processing DeleteTablet for tablet a238ad3d46054daa9435e9076ba8311d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:02 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:02.778010 13309 raft_consensus.cc:2804] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:02.801000 13308 ts_tablet_manager.cc:1929] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:46:02.775025 13371 raft_consensus.cc:515] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.801899 10869 tablet_service.cc:1511] Processing CreateTablet for tablet cec52957d4f5460d9f0846bc976926db (DEFAULT_TABLE table=default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.824648 13303 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: acquiring CA information for follower catalog manager: success
I20260809 01:46:02.826133 10846 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:45966 (request call id 56) took 1227 ms. Trace:
I20260809 01:46:02.826779 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d5a709cd86476594613e788a325094" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:46:02.830278 13314 leader_election.cc:290] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:02.831174 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa12353c25cb4da680d478ce104bc6ba" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:46:02.823026 11154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.10.8.67:33509 (request call id 242) took 1045 ms. Trace:
I20260809 01:46:02.837422 11154 rpcz_store.cc:276] 0809 01:46:01.777617 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.779402 (+ 1785us) service_pool.cc:224] Handling call
0809 01:46:01.814014 (+ 34612us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b5800000f98. stack: 00007fe4bc3e2cf3 00007fe4bc3e2bf0 00007fe4bbf90574 0000000000516de4 0000000000516d79 00007fe4bc29d981 00007fe4bc3dd521 00007fe4bc3dd3d2 00007fe4c5321fb9 00007fe4c4563640 00007fe4c462bebb 00007fe4c462bf41 00007fe4c45eddd0 00007fe4c45d8739 00007fe4c45d81e2 00007fe4c2c56d34
0809 01:46:01.842199 (+ 28185us) raft_consensus.cc:1471] Updating replica for 1 ops
0809 01:46:01.965919 (+123720us) raft_consensus.cc:1534] Early marking committed up to index 0
0809 01:46:01.965966 (+ 47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0809 01:46:01.966626 (+ 660us) log.cc:844] Serialized 26 byte log entry
0809 01:46:02.613719 (+647093us) raft_consensus.cc:1645] Marking committed up to 0
0809 01:46:02.669967 (+ 56248us) raft_consensus.cc:1697] Filling consensus response to leader.
0809 01:46:02.696653 (+ 26686us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0809 01:46:02.752236 (+ 55583us) raft_consensus.cc:1685] finished
0809 01:46:02.752256 (+ 20us) raft_consensus.cc:1691] UpdateReplicas() finished
0809 01:46:02.811624 (+ 59368us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":66839,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":216,"spinlock_wait_cycles":29659008,"thread_start_us":741648,"threads_started":2,"wal-append.queue_time_us":595337}
I20260809 01:46:02.842536 10596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_23821a5167f24f9882d9b3dc6edd65bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:02.843992 10596 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_23821a5167f24f9882d9b3dc6edd65bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:46:02.828539 10805 leader_election.cc:343] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.846184 10846 rpcz_store.cc:276] 0809 01:46:01.598476 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.598599 (+ 123us) service_pool.cc:224] Handling call
0809 01:46:02.825581 (+1226982us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:46:02.852437 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bc7dc3dfc2f4c1286ca24b8e3b20719" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
W20260809 01:46:02.853825 13342 log.cc:927] Time spent T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7: Append to log took a long time: real 0.076s user 0.000s sys 0.001s
I20260809 01:46:02.853844 10847 tablet_service.cc:1511] Processing CreateTablet for tablet d7296dd44f5b4f76aba38d2d9e0e6e85 (DEFAULT_TABLE table=default.Table_2708bfe75ad04882bc9a84d8e502c837 [id=8e5bd9b950c74620971c70cbfe31ed23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.856066 11395 raft_consensus.cc:3060] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:02.823262 13309 raft_consensus.cc:493] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:02.863875 13309 raft_consensus.cc:3060] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:02.867316 13371 leader_election.cc:290] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:02.874114 11554 tablet_bootstrap.cc:654] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:02.829397 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d5a709cd86476594613e788a325094" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:46:02.877633 10908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa12353c25cb4da680d478ce104bc6ba" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:46:02.878129 10908 raft_consensus.cc:3060] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:02.879257 13308 log.cc:1199] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/100ad529186e4821b85ba28996933a8a
I20260809 01:46:02.880393 13308 ts_tablet_manager.cc:1950] T 100ad529186e4821b85ba28996933a8a P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
W20260809 01:46:02.884459 11063 leader_election.cc:343] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 1bc7dc3dfc2f4c1286ca24b8e3b20719
I20260809 01:46:02.888581 11395 raft_consensus.cc:2468] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:46:02.895970 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 033eaf96ef5d4c6f87c28f70bde94ff0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:46:02.902000 10580 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 1c5897b881c448dab0cafe9238fa4bbb (table default.Table_80bdba205194482d96b26cb139375706 [id=b098ff5e09a0421ca14522917ab6e4f1], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1c5897b881c448dab0cafe9238fa4bbb consensus state has no leader
I20260809 01:46:02.902797 11553 tablet_bootstrap.cc:492] T e5af40994c204c4f82e36a53e00f2dfe P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:46:02.903802 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cec52957d4f5460d9f0846bc976926db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.910383 10847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7296dd44f5b4f76aba38d2d9e0e6e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.911747 13308 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 375) took 1525 ms. Trace:
I20260809 01:46:02.912096 11341 tablet_service.cc:1511] Processing CreateTablet for tablet cec52957d4f5460d9f0846bc976926db (DEFAULT_TABLE table=default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.912331 13308 rpcz_store.cc:276] 0809 01:46:01.386435 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.386588 (+ 153us) service_pool.cc:224] Handling call
0809 01:46:02.262850 (+876262us) ts_tablet_manager.cc:1151] Deleting tablet 100ad529186e4821b85ba28996933a8a
0809 01:46:02.262921 (+ 71us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:02.674444 (+411523us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:02.800884 (+126440us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:02.911628 (+110744us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":875589}
I20260809 01:46:02.913355 11336 tablet_service.cc:1511] Processing CreateTablet for tablet d7296dd44f5b4f76aba38d2d9e0e6e85 (DEFAULT_TABLE table=default.Table_2708bfe75ad04882bc9a84d8e502c837 [id=8e5bd9b950c74620971c70cbfe31ed23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.914522 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet 100ad529186e4821b85ba28996933a8a (table default.Table_3aab7a7876fe46caadaed8dde7387416 [id=67b5f34f642646319bbfa3de7ed8895d]) successfully deleted
I20260809 01:46:02.914716 13308 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 382) took 1257 ms. Trace:
I20260809 01:46:02.915251 13308 rpcz_store.cc:276] 0809 01:46:01.657579 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.657765 (+ 186us) service_pool.cc:224] Handling call
0809 01:46:02.914356 (+1256591us) ts_tablet_manager.cc:1151] Deleting tablet 14b1d03085eb4ab69789fed6fab882ae
0809 01:46:02.914405 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:02.914635 (+ 230us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1207854}
I20260809 01:46:02.916061 13371 raft_consensus.cc:493] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:02.916962 13308 tablet_replica.cc:333] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
W20260809 01:46:02.917122 10580 catalog_manager.cc:5017] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): delete failed for tablet 14b1d03085eb4ab69789fed6fab882ae because the tablet was not found. No further retry: Not found: Tablet not found: 14b1d03085eb4ab69789fed6fab882ae
I20260809 01:46:02.916746 13371 raft_consensus.cc:515] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:02.919852 13371 leader_election.cc:290] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:02.921192 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b334f049e24768ad2291326ae70121" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:46:02.749327 13319 raft_consensus.cc:493] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:02.923677 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 457ae94c3f9b4f4dbe8cd3101e4c578b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:02.923293 13319 raft_consensus.cc:515] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:46:02.929849 10805 leader_election.cc:343] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.931994 11553 tablet_bootstrap.cc:654] T e5af40994c204c4f82e36a53e00f2dfe P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
W20260809 01:46:02.903968 12870 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.124:33929, user_credentials={real_user=slave}} blocked reactor thread for 137987us
I20260809 01:46:02.932029 10586 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_67bd4213ff9c478b9a51efdfe77bcdb7" } new_table_name: "default.Table_6664c3771ae241d8ba912138ae117ef5" modify_external_catalogs: true
I20260809 01:46:02.937168 10589 master_service.cc:960] ReplaceTablet: received request to replace tablet 325e3d19b83545a197c2e09fc9643b5e from {username='slave'} at 127.0.0.1:37956
I20260809 01:46:02.945937 10588 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_267d92fa9322471a9e85b9bf80061ff3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:02.947633 10588 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_267d92fa9322471a9e85b9bf80061ff3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:02.954424 11127 tablet_service.cc:1511] Processing CreateTablet for tablet cec52957d4f5460d9f0846bc976926db (DEFAULT_TABLE table=default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 01:46:02.956070 11063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 70830.7us
I20260809 01:46:02.958932 11106 tablet_service.cc:1511] Processing CreateTablet for tablet d7296dd44f5b4f76aba38d2d9e0e6e85 (DEFAULT_TABLE table=default.Table_2708bfe75ad04882bc9a84d8e502c837 [id=8e5bd9b950c74620971c70cbfe31ed23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:02.961230 11156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b334f049e24768ad2291326ae70121" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:02.964599 11062 leader_election.cc:304] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:02.933928 10903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6c244108a4ff3a2df0585e5280294" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" is_pre_election: true
I20260809 01:46:02.968190 10903 raft_consensus.cc:2468] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:46:02.942565 13308 raft_consensus.cc:2243] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:02.889394 10908 raft_consensus.cc:2468] T fa12353c25cb4da680d478ce104bc6ba P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:46:02.971558 11063 leader_election.cc:304] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:02.971493 13308 raft_consensus.cc:2272] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260809 01:46:02.962679 10804 leader_election.cc:343] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:02.972242 10804 leader_election.cc:304] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:02.973968 13371 raft_consensus.cc:2749] T 34b334f049e24768ad2291326ae70121 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:02.974654 13308 ts_tablet_manager.cc:1916] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:02.976565 13289 raft_consensus.cc:2804] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:02.980450 13289 raft_consensus.cc:697] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:46:02.985030 13335 raft_consensus.cc:2804] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:02.986199 13335 raft_consensus.cc:493] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:02.986670 13335 raft_consensus.cc:3060] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:02.996541 13341 consensus_queue.cc:1048] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260809 01:46:03.004962 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cec52957d4f5460d9f0846bc976926db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:03.010002 11106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7296dd44f5b4f76aba38d2d9e0e6e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:03.017490 11107 tablet_service.cc:1511] Processing CreateTablet for tablet 52d4359daa9c408a9fad933406451e42 (DEFAULT_TABLE table=default.Table_483ea515938a4f9aa7347c3a8f948cf5 [id=9f1d5d1a3b40497aafa62c47703af9c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:03.019896 11107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52d4359daa9c408a9fad933406451e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:03.027086 13289 consensus_queue.cc:237] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.055388 13334 consensus_queue.cc:1048] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:46:03.063364 10855 tablet_service.cc:1511] Processing CreateTablet for tablet 52d4359daa9c408a9fad933406451e42 (DEFAULT_TABLE table=default.Table_483ea515938a4f9aa7347c3a8f948cf5 [id=9f1d5d1a3b40497aafa62c47703af9c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:03.064491 10855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52d4359daa9c408a9fad933406451e42. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:46:03.071657 13327 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.755s user 0.006s sys 0.047s
I20260809 01:46:03.075148 11114 tablet_service.cc:1511] Processing CreateTablet for tablet f2436d0015654dd996936824307a39c1 (DEFAULT_TABLE table=default.Table_49d42634313a4792b68ba9d39e4219f6 [id=55451619448c43f9b0da7f0ba4a6d94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:03.077325 11114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2436d0015654dd996936824307a39c1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:46:03.071976 13327 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s user 0.006s sys 0.047s
I20260809 01:46:02.972049 11289 leader_election.cc:304] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8; no voters:
I20260809 01:46:03.054394 11554 tablet_bootstrap.cc:492] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
W20260809 01:46:03.103966 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 132124us
I20260809 01:46:03.104316 11554 ts_tablet_manager.cc:1403] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.362s user 0.014s sys 0.007s
I20260809 01:46:03.105011 13237 raft_consensus.cc:2804] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:03.105485 13237 raft_consensus.cc:697] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:46:03.106269 13237 consensus_queue.cc:237] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.107623 11554 raft_consensus.cc:359] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.108194 13273 raft_consensus.cc:493] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:03.108613 13273 raft_consensus.cc:515] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.110914 13273 leader_election.cc:290] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:46:03.113538 11283 leader_election.cc:343] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:03.113685 13161 tablet_replica.cc:333] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:46:03.115464 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 033eaf96ef5d4c6f87c28f70bde94ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:03.118865 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cec52957d4f5460d9f0846bc976926db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:03.122838 11336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7296dd44f5b4f76aba38d2d9e0e6e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:03.124922 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 52d4359daa9c408a9fad933406451e42 (DEFAULT_TABLE table=default.Table_483ea515938a4f9aa7347c3a8f948cf5 [id=9f1d5d1a3b40497aafa62c47703af9c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:03.125896 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52d4359daa9c408a9fad933406451e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:03.108219 11554 raft_consensus.cc:385] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:03.130545 11554 raft_consensus.cc:740] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:03.131091 11554 consensus_queue.cc:260] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.132498 11554 ts_tablet_manager.cc:1434] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.028s user 0.002s sys 0.002s
W20260809 01:46:03.142701 11289 leader_election.cc:343] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 91941616dfdd4df3aaa7c8460876d273
I20260809 01:46:03.153367 10869 tablet_service.cc:1511] Processing CreateTablet for tablet f2436d0015654dd996936824307a39c1 (DEFAULT_TABLE table=default.Table_49d42634313a4792b68ba9d39e4219f6 [id=55451619448c43f9b0da7f0ba4a6d94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:03.154636 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2436d0015654dd996936824307a39c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:03.160161 10591 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_ad502d30876d40ad9da758848ef7e51a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:03.160620 10592 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_81380e3f093c48cb8811640969271318" } modify_external_catalogs: true
I20260809 01:46:03.161108 10592 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_81380e3f093c48cb8811640969271318" } modify_external_catalogs: true
W20260809 01:46:03.161824 10591 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ad502d30876d40ad9da758848ef7e51a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:03.165143 10597 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_32967e2b77ab462caff32362df47f2c6" } modify_external_catalogs: true
I20260809 01:46:03.165664 10597 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_32967e2b77ab462caff32362df47f2c6" } modify_external_catalogs: true
I20260809 01:46:03.168823 10598 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_81ae36e5d84645d9a61cf6a34b000deb" } new_table_name: "default.Table_843b166662904713a045a2a0cf7f8bb2" modify_external_catalogs: true
I20260809 01:46:03.169267 10596 master_service.cc:960] ReplaceTablet: received request to replace tablet 4071ee02730f42cea7bafe35aa4ca63d from {username='slave'} at 127.0.0.1:37956
I20260809 01:46:03.171988 13371 raft_consensus.cc:493] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:46:03.172669 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 122779us
I20260809 01:46:03.172492 13371 raft_consensus.cc:515] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.174355 13371 leader_election.cc:290] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:03.175830 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c87361b29664464f9fd73be974e2d3a0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:46:03.176077 13161 raft_consensus.cc:2243] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:03.176550 13161 raft_consensus.cc:2272] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 01:46:03.178104 10805 leader_election.cc:343] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:03.179415 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_9bcc86385a0840debdb02e97c0bd2144"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:03.181104 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9bcc86385a0840debdb02e97c0bd2144 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:46:03.193976 11062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 229436us
W20260809 01:46:03.203712 11111 consensus_meta.cc:287] T 8b4bbc6d4c594a76a3729d27926ae479 P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 0.552s user 0.006s sys 0.002s
I20260809 01:46:03.206769 10592 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet b5bd08670e2e40e0bb250d5e054a4116
W20260809 01:46:03.208588 10586 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3938bfd21b294fa6a25f0d8efc1ca29a consensus state has no leader
I20260809 01:46:03.213373 10595 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet 34616fd5895945b3aa896d5a8ee87249 (table default.Table_f67555499dc34738b62598f70a738d8d [id=946eac74789f4001b253e66f9ff48fe8]). Expected version 1 got 0
I20260809 01:46:03.215303 11123 tablet_service.cc:1558] Processing DeleteTablet for tablet b5bd08670e2e40e0bb250d5e054a4116 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:03 UTC) from {username='slave'} at 127.0.0.1:56566
W20260809 01:46:03.216049 11063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 44749.2us
I20260809 01:46:03.182410 13161 ts_tablet_manager.cc:1916] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:03.185884 13083 raft_consensus.cc:493] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:03.191866 13319 leader_election.cc:290] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:46:03.192907 11289 leader_election.cc:343] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: 9fffa48ccf58476da79dccd9c008b67c
I20260809 01:46:03.200423 13382 raft_consensus.cc:493] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:03.221786 13382 raft_consensus.cc:515] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.226373 13303 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 01:46:03.235570 11349 tablet_service.cc:1511] Processing CreateTablet for tablet f2436d0015654dd996936824307a39c1 (DEFAULT_TABLE table=default.Table_49d42634313a4792b68ba9d39e4219f6 [id=55451619448c43f9b0da7f0ba4a6d94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:03.237039 11349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2436d0015654dd996936824307a39c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:03.241645 13382 leader_election.cc:290] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:03.252954 13161 ts_tablet_manager.cc:1929] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:46:03.258565 11345 tablet_service.cc:1558] Processing DeleteTablet for tablet b5bd08670e2e40e0bb250d5e054a4116 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:03 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:03.264937 13161 log.cc:1199] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/1bc7dc3dfc2f4c1286ca24b8e3b20719
I20260809 01:46:03.266276 13161 ts_tablet_manager.cc:1950] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:46:03.280035 11553 tablet_bootstrap.cc:492] T e5af40994c204c4f82e36a53e00f2dfe P b5117038a52a45b0aea3d30ca9cfcbe8: No bootstrap required, opened a new log
I20260809 01:46:03.280454 11553 ts_tablet_manager.cc:1403] T e5af40994c204c4f82e36a53e00f2dfe P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent bootstrapping tablet: real 0.378s user 0.006s sys 0.018s
I20260809 01:46:03.275933 13083 raft_consensus.cc:515] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.283094 13083 leader_election.cc:290] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:03.285495 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750aa437ef594587a512a9c9c8278aa8" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:46:03.300436 11345 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fffa48ccf58476da79dccd9c008b67c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:53 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:03.302914 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750aa437ef594587a512a9c9c8278aa8" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:03.304533 11154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c87361b29664464f9fd73be974e2d3a0" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:46:03.309970 10805 leader_election.cc:343] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:46:03.311210 11289 leader_election.cc:343] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: bb7a8b6d06064addb02979a7feedf756
I20260809 01:46:03.317260 10861 tablet_service.cc:1558] Processing DeleteTablet for tablet b5bd08670e2e40e0bb250d5e054a4116 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:03 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:03.319516 11111 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 409) took 1005 ms. Trace:
I20260809 01:46:03.319820 11111 rpcz_store.cc:276] 0809 01:46:02.314561 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:02.322772 (+ 8211us) service_pool.cc:224] Handling call
0809 01:46:02.328762 (+ 5990us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:46:02.328843 (+ 81us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:46:02.582384 (+253541us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:03.223206 (+640822us) tablet_replica.cc:206] Creating consensus instance
0809 01:46:03.319466 (+ 96260us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260809 01:46:03.325186 10598 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a1b924fd94b44d1e85480a2feeca57e6 has no consensus state
I20260809 01:46:03.350031 11554 tablet_bootstrap.cc:492] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:46:03.352550 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 388) took 3623 ms. Trace:
I20260809 01:46:03.352821 13161 rpcz_store.cc:276] 0809 01:45:59.729320 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:59.729594 (+ 274us) service_pool.cc:224] Handling call
0809 01:46:02.789235 (+3059641us) ts_tablet_manager.cc:1151] Deleting tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719
0809 01:46:03.113568 (+324333us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:03.234274 (+120706us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:03.252851 (+ 18577us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:03.352495 (+ 99644us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":137289,"tablet-delete.queue_time_us":3038726}
I20260809 01:46:03.354069 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 389) took 3553 ms. Trace:
I20260809 01:46:03.354364 13161 rpcz_store.cc:276] 0809 01:45:59.800450 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:59.800586 (+ 136us) service_pool.cc:224] Handling call
0809 01:46:03.353771 (+3553185us) ts_tablet_manager.cc:1151] Deleting tablet 14b1d03085eb4ab69789fed6fab882ae
0809 01:46:03.353807 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:03.354021 (+ 214us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3552678}
I20260809 01:46:03.356302 11554 tablet_bootstrap.cc:654] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:03.356748 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 1bc7dc3dfc2f4c1286ca24b8e3b20719 (table default.Table_ae59ae0a1b7d4312856fd310b0c6c8b7 [id=96595f8bb6b84744bf7fc74f074a51ba]) successfully deleted
I20260809 01:46:03.357105 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 390) took 3241 ms. Trace:
W20260809 01:46:03.357401 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 14b1d03085eb4ab69789fed6fab882ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14b1d03085eb4ab69789fed6fab882ae already in progress: creating tablet
I20260809 01:46:03.357401 13161 rpcz_store.cc:276] 0809 01:46:00.116004 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:00.125668 (+ 9664us) service_pool.cc:224] Handling call
0809 01:46:03.355074 (+3229406us) ts_tablet_manager.cc:1151] Deleting tablet fa29cba15cc247debfb8e1907ecd0412
0809 01:46:03.355105 (+ 31us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:03.357044 (+ 1939us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3228818}
I20260809 01:46:03.358853 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 395) took 1972 ms. Trace:
I20260809 01:46:03.359117 13161 rpcz_store.cc:276] 0809 01:46:01.386035 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.386220 (+ 185us) service_pool.cc:224] Handling call
0809 01:46:03.358448 (+1972228us) ts_tablet_manager.cc:1151] Deleting tablet 100ad529186e4821b85ba28996933a8a
0809 01:46:03.358485 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:03.358809 (+ 324us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1971741}
W20260809 01:46:03.359503 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet fa29cba15cc247debfb8e1907ecd0412 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa29cba15cc247debfb8e1907ecd0412 already in progress: creating tablet
W20260809 01:46:03.360097 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 100ad529186e4821b85ba28996933a8a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 100ad529186e4821b85ba28996933a8a already in progress: creating tablet
I20260809 01:46:03.360582 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 394) took 1979 ms. Trace:
I20260809 01:46:03.361086 13161 rpcz_store.cc:276] 0809 01:46:01.380927 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.385755 (+ 4828us) service_pool.cc:224] Handling call
0809 01:46:03.360269 (+1974514us) ts_tablet_manager.cc:1151] Deleting tablet 31874b7e70a847a4885bd09da40a6902
0809 01:46:03.360304 (+ 35us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:03.360537 (+ 233us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1933018}
I20260809 01:46:03.362324 13161 tablet_replica.cc:333] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:46:03.363023 13161 raft_consensus.cc:2243] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:03.363415 13161 raft_consensus.cc:2272] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:03.364990 13161 ts_tablet_manager.cc:1916] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:03.368551 11395 raft_consensus.cc:3060] T 7c7a511512184f53921cea081f82aa37 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:03.374822 11153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fffa48ccf58476da79dccd9c008b67c" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:46:03.375758 10804 leader_election.cc:343] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:03.375922 11149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7a8b6d06064addb02979a7feedf756" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:03.376152 10804 leader_election.cc:304] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:03.376525 11149 raft_consensus.cc:2468] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
W20260809 01:46:03.379179 11283 leader_election.cc:343] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: Tablet not RUNNING: STOPPED
I20260809 01:46:03.379545 11283 leader_election.cc:304] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:46:03.380523 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 31874b7e70a847a4885bd09da40a6902 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31874b7e70a847a4885bd09da40a6902 already in progress: creating tablet
W20260809 01:46:03.380550 11283 leader_election.cc:343] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: 91941616dfdd4df3aaa7c8460876d273
I20260809 01:46:03.380903 11283 leader_election.cc:304] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:46:03.381389 11283 raft_consensus.cc:2686] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 01:46:03.382630 11283 leader_election.cc:304] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
W20260809 01:46:03.377336 10804 leader_election.cc:343] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:03.384022 13319 raft_consensus.cc:2749] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:03.384359 10804 leader_election.cc:304] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:03.385474 13319 raft_consensus.cc:2804] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:03.389955 13319 raft_consensus.cc:493] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:03.390658 13319 raft_consensus.cc:3060] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:03.394795 11116 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 403) took 1254 ms. Trace:
I20260809 01:46:03.395018 11124 tablet_service.cc:1511] Processing CreateTablet for tablet a44ae38eea304dbb9fa511f6311b0f41 (DEFAULT_TABLE table=default.Table_88a5a7cfe7174a30bc65b1943fc5e8b2 [id=3f7b4c833dcf4e6683d21a7242538f77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:03.395113 11116 rpcz_store.cc:276] 0809 01:46:02.140662 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:02.178866 (+ 38204us) service_pool.cc:224] Handling call
0809 01:46:02.265837 (+ 86971us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:46:02.265951 (+ 114us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:46:03.318465 (+1052514us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:03.394308 (+ 75843us) tablet_replica.cc:206] Creating consensus instance
0809 01:46:03.394747 (+ 439us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:46:03.405357 13415 consensus_queue.cc:1048] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 01:46:03.407202 13309 consensus_meta.cc:287] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 0.543s user 0.010s sys 0.001s
I20260809 01:46:03.369948 13310 ts_tablet_manager.cc:1929] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:46:03.420868 11361 tablet_service.cc:1511] Processing CreateTablet for tablet a44ae38eea304dbb9fa511f6311b0f41 (DEFAULT_TABLE table=default.Table_88a5a7cfe7174a30bc65b1943fc5e8b2 [id=3f7b4c833dcf4e6683d21a7242538f77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:03.422425 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a44ae38eea304dbb9fa511f6311b0f41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:03.434553 13371 raft_consensus.cc:2749] T c87361b29664464f9fd73be974e2d3a0 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:03.410738 13083 raft_consensus.cc:2749] T 750aa437ef594587a512a9c9c8278aa8 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:03.448601 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a44ae38eea304dbb9fa511f6311b0f41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:03.460273 13161 ts_tablet_manager.cc:1929] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:46:03.462209 10597 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 0e011390c1ef43da84445038c66d9970
I20260809 01:46:03.463037 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 325e3d19b83545a197c2e09fc9643b5e
I20260809 01:46:03.463382 10599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_8e7ff59db91443ea8b96438da5589a9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:03.464845 10599 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8e7ff59db91443ea8b96438da5589a9a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:03.465373 10589 catalog_manager.cc:6807] ReplaceTablet: tablet 325e3d19b83545a197c2e09fc9643b5e deleted and replaced by tablet 465ecacad411445e8391f6396a8db861
I20260809 01:46:03.469784 10598 catalog_manager.cc:5811] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495) does not have the latest schema for tablet 3e03033672e947eab8cc143263281133 (table default.Table_109c891eea264584b9e1ceab647f32aa [id=24d2e675d969421fba4647380880f606]). Expected version 3 got 0
I20260809 01:46:03.470768 10598 catalog_manager.cc:5719] T 7c7a511512184f53921cea081f82aa37 P 33b5652538e64ed58a09993d3188f8d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66). New cstate: current_term: 1 leader_uuid: "33b5652538e64ed58a09993d3188f8d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } }
I20260809 01:46:03.475992 10589 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_79eb7d6b90da4bccbad721f2e840dd31" } modify_external_catalogs: true
I20260809 01:46:03.476689 10589 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_79eb7d6b90da4bccbad721f2e840dd31" } modify_external_catalogs: true
W20260809 01:46:03.477552 10804 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 100217us
I20260809 01:46:03.479336 10861 tablet_service.cc:1511] Processing CreateTablet for tablet a44ae38eea304dbb9fa511f6311b0f41 (DEFAULT_TABLE table=default.Table_88a5a7cfe7174a30bc65b1943fc5e8b2 [id=3f7b4c833dcf4e6683d21a7242538f77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:03.475519 10595 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_5306ac111c6b42caa5357a817257dda3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:03.480643 10847 tablet_service.cc:1558] Processing DeleteTablet for tablet 325e3d19b83545a197c2e09fc9643b5e with delete_type TABLET_DATA_DELETED (replaced by tablet 465ecacad411445e8391f6396a8db861) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:03.492398 10597 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_a3fe3ca9a3c54ee9ad390799d0022776" } new_table_name: "default.Table_1159d2f595e04cca888bdf344fc28a24" modify_external_catalogs: true
W20260809 01:46:03.407123 10901 thread.cc:641] tablet-copy [worker] (thread pool) Time spent creating pthread: real 0.618s user 0.010s sys 0.008s
W20260809 01:46:03.485384 10595 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5306ac111c6b42caa5357a817257dda3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:03.500165 13398 tablet.cc:1724] T e3684fa56f9549f492a96a5c765eda78 P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260809 01:46:03.431365 11551 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.611s user 0.008s sys 0.010s
W20260809 01:46:03.519464 11551 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.699s user 0.009s sys 0.010s
I20260809 01:46:03.519325 11116 tablet_service.cc:1558] Processing DeleteTablet for tablet 325e3d19b83545a197c2e09fc9643b5e with delete_type TABLET_DATA_DELETED (replaced by tablet 465ecacad411445e8391f6396a8db861) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:03.524808 11553 raft_consensus.cc:359] T e5af40994c204c4f82e36a53e00f2dfe P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.525465 11553 raft_consensus.cc:385] T e5af40994c204c4f82e36a53e00f2dfe P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:03.525835 11553 raft_consensus.cc:740] T e5af40994c204c4f82e36a53e00f2dfe P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117038a52a45b0aea3d30ca9cfcbe8, State: Initialized, Role: FOLLOWER
W20260809 01:46:03.527207 13399 log.cc:927] Time spent T 325e3d19b83545a197c2e09fc9643b5e P b5117038a52a45b0aea3d30ca9cfcbe8: Append to log took a long time: real 0.089s user 0.000s sys 0.000s
I20260809 01:46:03.407581 13309 raft_consensus.cc:515] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.404695 13363 tablet.cc:1724] T e3684fa56f9549f492a96a5c765eda78 P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260809 01:46:03.540020 11062 consensus_peers.cc:597] T 325e3d19b83545a197c2e09fc9643b5e P 33b5652538e64ed58a09993d3188f8d7 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260809 01:46:03.507637 10901 thread.cc:608] tablet-copy [worker] (thread pool) Time spent starting thread: real 0.718s user 0.010s sys 0.008s
I20260809 01:46:03.551070 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a44ae38eea304dbb9fa511f6311b0f41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:03.560110 10592 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37956:
table { table_name: "default.Table_e09e4f1ce6624e9b91ade11edfcb3e80" } new_table_name: "default.Table_70c1d2858fbf4a54bf19133195af09d3" modify_external_catalogs: true
I20260809 01:46:03.562156 10586 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37956:
name: "default.Table_b786ef3b79564857bf001e731c82c71a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:03.562621 11119 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56566 (request call id 402) took 1439 ms. Trace:
W20260809 01:46:03.563638 10586 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b786ef3b79564857bf001e731c82c71a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:03.522349 13310 log.cc:1199] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/9fffa48ccf58476da79dccd9c008b67c
I20260809 01:46:03.575939 11551 tablet_bootstrap.cc:492] T 67b31998b65e4264b16c0d73dbcfc73b P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:46:03.576342 11551 ts_tablet_manager.cc:1403] T 67b31998b65e4264b16c0d73dbcfc73b P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 1.123s user 0.019s sys 0.010s
I20260809 01:46:03.596243 13310 ts_tablet_manager.cc:1950] T 9fffa48ccf58476da79dccd9c008b67c P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:46:03.602020 11155 raft_consensus.cc:2393] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b5117038a52a45b0aea3d30ca9cfcbe8 in current term 1: Already voted for candidate 33b5652538e64ed58a09993d3188f8d7 in this term.
I20260809 01:46:03.468367 13319 raft_consensus.cc:515] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.612661 13309 leader_election.cc:290] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:03.473153 13418 tablet.cc:1724] T e3684fa56f9549f492a96a5c765eda78 P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 01:46:03.484406 11554 tablet_bootstrap.cc:492] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:03.614881 11554 ts_tablet_manager.cc:1403] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.265s user 0.009s sys 0.007s
I20260809 01:46:03.622316 11554 raft_consensus.cc:359] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:46:03.624318 13335 consensus_meta.cc:287] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7: Time spent flushing consensus metadata: real 0.637s user 0.000s sys 0.020s
I20260809 01:46:03.470125 11336 tablet_service.cc:1558] Processing DeleteTablet for tablet 325e3d19b83545a197c2e09fc9643b5e with delete_type TABLET_DATA_DELETED (replaced by tablet 465ecacad411445e8391f6396a8db861) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:03.628211 13319 leader_election.cc:290] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:03.650053 10804 leader_election.cc:304] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
I20260809 01:46:03.640560 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d5a709cd86476594613e788a325094" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:46:03.646391 11554 raft_consensus.cc:385] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:03.652164 11551 raft_consensus.cc:359] T 67b31998b65e4264b16c0d73dbcfc73b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.652810 11554 raft_consensus.cc:740] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:03.653035 11551 raft_consensus.cc:385] T 67b31998b65e4264b16c0d73dbcfc73b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:03.653632 11551 raft_consensus.cc:740] T 67b31998b65e4264b16c0d73dbcfc73b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
W20260809 01:46:03.654369 11062 leader_election.cc:343] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:03.653504 11554 consensus_queue.cc:260] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.460659 13161 log.cc:1199] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/91941616dfdd4df3aaa7c8460876d273
I20260809 01:46:03.667153 11554 ts_tablet_manager.cc:1434] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.052s user 0.003s sys 0.004s
I20260809 01:46:03.668111 11554 tablet_bootstrap.cc:492] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:46:03.664304 10273 master-stress-test.cc:579] Killing test thread 1/10
I20260809 01:46:03.563023 11119 rpcz_store.cc:276] 0809 01:46:02.122993 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:02.135714 (+ 12721us) service_pool.cc:224] Handling call
0809 01:46:02.324137 (+188423us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b5800000f98. stack: 00007fe4bc3e2cf3 00007fe4bc3e2bf0 00007fe4bbf90574 0000000000516de4 0000000000516d79 00007fe4bc29d981 00007fe4bc3ddc68 00007fe4bc3ddb8b 00007fe4c534b1d9 00007fe4c45634e8 00007fe4c4625990 00007fe4c45d4210 00007fe4c2b60bf4 00007fe4c2b60b82 00007fe4c2b60ab1 00007fe4c2b60a2c
0809 01:46:02.324178 (+ 41us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0809 01:46:02.324310 (+ 132us) ts_tablet_manager.cc:682] Creating new metadata...
0809 01:46:03.200613 (+876303us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:03.541175 (+340562us) tablet_replica.cc:206] Creating consensus instance
0809 01:46:03.562557 (+ 21382us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13009664}
I20260809 01:46:03.677711 13161 ts_tablet_manager.cc:1950] T 91941616dfdd4df3aaa7c8460876d273 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:46:03.679636 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 401) took 2075 ms. Trace:
I20260809 01:46:03.680153 13428 raft_consensus.cc:493] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 629f002048f64491afdd57f72a466e59)
I20260809 01:46:03.680769 13428 raft_consensus.cc:515] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:46:03.678059 11553 consensus_queue.cc:260] T e5af40994c204c4f82e36a53e00f2dfe P b5117038a52a45b0aea3d30ca9cfcbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.689088 11553 ts_tablet_manager.cc:1434] T e5af40994c204c4f82e36a53e00f2dfe P b5117038a52a45b0aea3d30ca9cfcbe8: Time spent starting tablet: real 0.408s user 0.006s sys 0.003s
I20260809 01:46:03.693631 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 453 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:46:03.693565 13428 leader_election.cc:290] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:46:03.685184 13371 raft_consensus.cc:2749] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 01:46:03.624670 13335 raft_consensus.cc:515] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.718392 13335 leader_election.cc:290] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:03.687443 13310 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 390) took 1997 ms. Trace:
I20260809 01:46:03.719568 13310 rpcz_store.cc:276] 0809 01:46:01.689940 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.690371 (+ 431us) service_pool.cc:224] Handling call
0809 01:46:01.751094 (+ 60723us) ts_tablet_manager.cc:1151] Deleting tablet 9fffa48ccf58476da79dccd9c008b67c
0809 01:46:01.754436 (+ 3342us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:02.180717 (+426281us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:03.366441 (+1185724us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:03.681891 (+315450us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25641,"tablet-delete.queue_time_us":1615}
I20260809 01:46:03.721612 13310 tablet_replica.cc:333] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:46:03.722782 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 9fffa48ccf58476da79dccd9c008b67c (table default.Table_3227dde355ef4460a73ac13726bf7249 [id=3e51bbb9edf142ecabc9c92777fd80b4]) successfully deleted
I20260809 01:46:03.722958 13310 raft_consensus.cc:2243] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:03.724303 13310 raft_consensus.cc:2272] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:03.728057 13310 ts_tablet_manager.cc:1916] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:03.728119 13308 ts_tablet_manager.cc:1929] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:46:03.728497 13308 log.cc:1199] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/b3b02bd8c93b43bf90ff0192a9541aae
I20260809 01:46:03.726519 11551 consensus_queue.cc:260] T 67b31998b65e4264b16c0d73dbcfc73b P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.729230 13308 ts_tablet_manager.cc:1950] T b3b02bd8c93b43bf90ff0192a9541aae P b5117038a52a45b0aea3d30ca9cfcbe8: Deleting consensus metadata
I20260809 01:46:03.731468 11551 ts_tablet_manager.cc:1434] T 67b31998b65e4264b16c0d73dbcfc73b P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.155s user 0.006s sys 0.002s
I20260809 01:46:03.732347 11551 tablet_bootstrap.cc:492] T 81da8d08131d4a3ba636ecfc7a7d2951 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:46:03.759228 13083 raft_consensus.cc:493] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:03.759899 13083 raft_consensus.cc:515] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:03.771588 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9a0fc14dd684ccaa19d51261b9c610e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:46:03.773586 10804 leader_election.cc:343] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:03.784137 10903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6c244108a4ff3a2df0585e5280294" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:46:03.784864 10903 raft_consensus.cc:3060] T afb6c244108a4ff3a2df0585e5280294 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:03.787140 11155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7a8b6d06064addb02979a7feedf756" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:46:03.787834 11155 raft_consensus.cc:3060] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:46:03.788754 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 76351.1us
W20260809 01:46:03.789511 11289 leader_election.cc:343] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Not found: Tablet not found: bb7a8b6d06064addb02979a7feedf756
I20260809 01:46:03.800634 13308 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45982 (request call id 383) took 2141 ms. Trace:
I20260809 01:46:03.757107 13161 rpcz_store.cc:276] 0809 01:46:01.604716 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.604849 (+ 133us) service_pool.cc:224] Handling call
0809 01:46:03.362160 (+1757311us) ts_tablet_manager.cc:1151] Deleting tablet 91941616dfdd4df3aaa7c8460876d273
0809 01:46:03.362199 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:03.426445 (+ 64246us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:03.460113 (+ 33668us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:03.679584 (+219471us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1756669}
I20260809 01:46:03.805636 13308 rpcz_store.cc:276] 0809 01:46:01.659203 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.659382 (+ 179us) service_pool.cc:224] Handling call
0809 01:46:02.916697 (+1257315us) ts_tablet_manager.cc:1151] Deleting tablet b3b02bd8c93b43bf90ff0192a9541aae
0809 01:46:02.916809 (+ 112us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:03.363007 (+446198us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:03.727986 (+364979us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:03.800488 (+ 72502us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1210027}
I20260809 01:46:03.808971 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9a0fc14dd684ccaa19d51261b9c610e" candidate_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:46:03.811811 13363 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:56566 (request call id 392) took 2065 ms. Trace:
I20260809 01:46:03.809958 13161 tablet_replica.cc:333] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:46:03.829555 13161 raft_consensus.cc:2243] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:03.833894 13395 tablet.cc:1724] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:46:03.835974 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 91941616dfdd4df3aaa7c8460876d273 (table default.Table_58a6cb5f8b8043519208cd2d2f222940 [id=92630f3e58394023a931a5e2c40e8591]) successfully deleted
I20260809 01:46:03.848650 13382 consensus_queue.cc:1048] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 01:46:03.810423 10805 leader_election.cc:343] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:03.859576 10805 leader_election.cc:304] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5117038a52a45b0aea3d30ca9cfcbe8; no voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316
W20260809 01:46:03.860224 10805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.67:40895, user_credentials={real_user=slave}} blocked reactor thread for 49834.8us
I20260809 01:46:03.861896 13371 raft_consensus.cc:2749] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:03.811770 13083 leader_election.cc:290] T f9a0fc14dd684ccaa19d51261b9c610e P b5117038a52a45b0aea3d30ca9cfcbe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895)
I20260809 01:46:03.813290 11553 tablet_bootstrap.cc:492] T 778d1582d67d44b9bc25a1db6d819ccb P b5117038a52a45b0aea3d30ca9cfcbe8: Bootstrap starting.
I20260809 01:46:03.797138 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d5a709cd86476594613e788a325094" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8"
I20260809 01:46:03.870271 10897 raft_consensus.cc:2393] T 82d5a709cd86476594613e788a325094 P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 33b5652538e64ed58a09993d3188f8d7 in current term 1: Already voted for candidate b5117038a52a45b0aea3d30ca9cfcbe8 in this term.
I20260809 01:46:03.881990 13398 tablet.cc:1724] T 63c1a610663a4a2fa58f6cdc5f8d86ce P b5117038a52a45b0aea3d30ca9cfcbe8: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 01:46:03.886250 11063 leader_election.cc:304] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7; no voters: 4f60c9ac9414403d9ea5a97b103c2316, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:03.887419 13334 raft_consensus.cc:2749] T 82d5a709cd86476594613e788a325094 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 01:46:03.890302 10580 catalog_manager.cc:5050] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): tablet b3b02bd8c93b43bf90ff0192a9541aae (table default.Table_085fd4536b524eb9aacb353d95ffa166 [id=c0797cd687f2480087ce8379df004967]) successfully deleted
I20260809 01:46:03.892360 13161 raft_consensus.cc:2272] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:03.898165 11155 raft_consensus.cc:2468] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
W20260809 01:46:03.902953 10581 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet cb13f4c7b07348cba67b6b43a9b154bd (table default.Table_a3fe3ca9a3c54ee9ad390799d0022776 [id=393bf4cab7eb48859853968aeae87b42], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cb13f4c7b07348cba67b6b43a9b154bd consensus state has no leader
I20260809 01:46:03.903542 11553 tablet_bootstrap.cc:654] T 778d1582d67d44b9bc25a1db6d819ccb P b5117038a52a45b0aea3d30ca9cfcbe8: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:03.812405 13363 rpcz_store.cc:276] 0809 01:46:01.746358 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.747563 (+ 1205us) service_pool.cc:224] Handling call
0809 01:46:03.811706 (+2064143us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:46:01.816487 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0809 01:46:01.816839 (+ 352us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 01:46:01.816869 (+ 30us) alter_schema_op.cc:66] Acquired schema lock
0809 01:46:01.816927 (+ 58us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0809 01:46:01.903612 (+ 86685us) spinlock_profiling.cc:243] Waited 65.7 ms on lock 0x7b2c00004d28. stack: 00007fe4bc3e2cf3 00007fe4bc3e2bf0 00007fe4bbf90574 0000000000516de4 0000000000516d79 00007fe4c2782c71 00007fe4c2791c6a 00007fe4c27918f3 00007fe4c2791891 00007fe4c35b5ce2 00007fe4c35b5abd 00007fe4c3a05c23 00007fe4c3a05738 00007fe4c3a0a3d1 00007fe4c3a0a389 00007fe4c3a0a319
0809 01:46:01.903722 (+ 110us) alter_schema_op.cc:128] START. Timestamp: 1786239961882700
0809 01:46:01.903759 (+ 37us) op_driver.cc:348] REPLICATION: starting
0809 01:46:01.904552 (+ 793us) log.cc:844] Serialized 173 byte log entry
0809 01:46:02.872136 (+967584us) op_driver.cc:464] REPLICATION: finished
0809 01:46:03.022856 (+150720us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b54002047a0. stack: 00007fe4bc3e2cf3 00007fe4bc3e2bf0 00007fe4bbf90574 0000000000516de4 0000000000516d79 00007fe4c34e3951 00007fe4c34f019e 00007fe4c34fad10 00007fe4c34facb8 00007fe4c34fac38 00007fe4c34fabf3 00007fe4c34f9e7c 00007fe4c5398a16 00007fe4c53931c8 00007fe4c3536bab 00007fe4c35396a0
0809 01:46:03.404541 (+381685us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0809 01:46:03.618001 (+213460us) log.cc:844] Serialized 5 byte log entry
0809 01:46:03.737032 (+119031us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:03.803476 (+ 66444us) log.cc:844] Serialized 14 byte log entry
0809 01:46:03.811465 (+ 7989us) alter_schema_op.cc:73] Released schema lock
0809 01:46:03.811516 (+ 51us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":532174,"prepare.queue_time_us":66702,"prepare.run_cpu_time_us":81511,"prepare.run_wall_time_us":623687,"replication_time_us":968109,"spinlock_wait_cycles":163011328,"thread_start_us":1206227,"threads_started":3,"wal-append.queue_time_us":574475}]]}
I20260809 01:46:03.911140 10589 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet b93faf596f3c4dd286e600013c8db001
I20260809 01:46:03.915146 13308 tablet_replica.cc:333] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8: stopping tablet replica
I20260809 01:46:03.922417 13161 ts_tablet_manager.cc:1916] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:03.917022 13308 raft_consensus.cc:2243] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:03.931388 13308 raft_consensus.cc:2272] T 68f022e6a57a4ec0a5a8c33159e97a8e P b5117038a52a45b0aea3d30ca9cfcbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:03.931705 10596 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: Sending DeleteTablet for 3 replicas of tablet 4071ee02730f42cea7bafe35aa4ca63d
W20260809 01:46:03.932063 10597 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cb13f4c7b07348cba67b6b43a9b154bd consensus state has no leader
I20260809 01:46:03.933699 10596 catalog_manager.cc:6807] ReplaceTablet: tablet 4071ee02730f42cea7bafe35aa4ca63d deleted and replaced by tablet 9f485597f2c84b45929686f6edd856d4
I20260809 01:46:03.935004 11106 tablet_service.cc:1558] Processing DeleteTablet for tablet 4071ee02730f42cea7bafe35aa4ca63d with delete_type TABLET_DATA_DELETED (replaced by tablet 9f485597f2c84b45929686f6edd856d4) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:03.937428 13395 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:56566 (request call id 401) took 1831 ms. Trace:
I20260809 01:46:03.937798 13395 rpcz_store.cc:276] 0809 01:46:02.105836 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:02.106195 (+ 359us) service_pool.cc:224] Handling call
0809 01:46:03.937368 (+1831173us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0809 01:46:02.181558 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0809 01:46:02.298343 (+116785us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 01:46:02.298394 (+ 51us) alter_schema_op.cc:66] Acquired schema lock
0809 01:46:02.298518 (+ 124us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0809 01:46:02.298679 (+ 161us) alter_schema_op.cc:128] START. Timestamp: 1786239962298520
0809 01:46:02.298696 (+ 17us) op_driver.cc:348] REPLICATION: starting
0809 01:46:02.309977 (+ 11281us) log.cc:844] Serialized 173 byte log entry
0809 01:46:03.832791 (+1522814us) op_driver.cc:464] REPLICATION: finished
0809 01:46:03.833683 (+ 892us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0809 01:46:03.836224 (+ 2541us) log.cc:844] Serialized 5 byte log entry
0809 01:46:03.935922 (+ 99698us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:03.936592 (+ 670us) log.cc:844] Serialized 14 byte log entry
0809 01:46:03.937187 (+ 595us) alter_schema_op.cc:73] Released schema lock
0809 01:46:03.937251 (+ 64us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":582,"prepare.queue_time_us":35626,"prepare.run_cpu_time_us":57074,"prepare.run_wall_time_us":952311,"raft.queue_time_us":124,"raft.run_cpu_time_us":30,"raft.run_wall_time_us":31,"replication_time_us":1533943,"spinlock_wait_cycles":719232,"thread_start_us":1446876,"threads_started":2,"wal-append.queue_time_us":788049}]]}
I20260809 01:46:03.941676 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 453 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:46:03.944509 12870 leader_election.cc:304] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bc72deb1d2402cabd817dfe3b27e88; no voters: 09a73ee8e8ec496a984efcab27715506, 629f002048f64491afdd57f72a466e59
I20260809 01:46:03.946462 13428 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 01:46:03.957736 11551 tablet_bootstrap.cc:654] T 81da8d08131d4a3ba636ecfc7a7d2951 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:03.962618 10855 tablet_service.cc:1558] Processing DeleteTablet for tablet b93faf596f3c4dd286e600013c8db001 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:03 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:03.962646 11124 tablet_service.cc:1558] Processing DeleteTablet for tablet b93faf596f3c4dd286e600013c8db001 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:03 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:03.968542 10855 tablet_service.cc:1558] Processing DeleteTablet for tablet 4071ee02730f42cea7bafe35aa4ca63d with delete_type TABLET_DATA_DELETED (replaced by tablet 9f485597f2c84b45929686f6edd856d4) from {username='slave'} at 127.0.0.1:45982
W20260809 01:46:03.973865 13424 log.cc:927] Time spent T 7c7a511512184f53921cea081f82aa37 P 4f60c9ac9414403d9ea5a97b103c2316: Append to log took a long time: real 0.185s user 0.002s sys 0.000s
I20260809 01:46:04.003204 11554 tablet_bootstrap.cc:654] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
W20260809 01:46:04.005636 10592 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 23f97a412594405bb5dca58e58f5dbc2 consensus state has no leader
W20260809 01:46:04.005671 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 210048us
I20260809 01:46:04.008677 11283 leader_election.cc:304] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters: b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:04.010947 13430 raft_consensus.cc:2804] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:04.013201 13430 raft_consensus.cc:697] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:46:04.014581 13430 consensus_queue.cc:237] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:46:04.042558 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 252426us
I20260809 01:46:04.056651 10855 tablet_service.cc:1558] Processing DeleteTablet for tablet 750aa437ef594587a512a9c9c8278aa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:47 UTC) from {username='slave'} at 127.0.0.1:45982
I20260809 01:46:04.074637 13310 ts_tablet_manager.cc:1929] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260809 01:46:04.075132 11283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 53549.6us
I20260809 01:46:04.080103 10273 master-stress-test.cc:581] Killed test thread 1/10
I20260809 01:46:04.080411 10273 master-stress-test.cc:579] Killing test thread 2/10
I20260809 01:46:04.080830 10273 master-stress-test.cc:581] Killed test thread 2/10
I20260809 01:46:04.081103 10273 master-stress-test.cc:579] Killing test thread 3/10
I20260809 01:46:04.081501 10273 master-stress-test.cc:581] Killed test thread 3/10
I20260809 01:46:04.081773 10273 master-stress-test.cc:579] Killing test thread 4/10
I20260809 01:46:04.082091 10273 master-stress-test.cc:581] Killed test thread 4/10
I20260809 01:46:04.082350 10273 master-stress-test.cc:579] Killing test thread 5/10
I20260809 01:46:04.082727 10273 master-stress-test.cc:581] Killed test thread 5/10
I20260809 01:46:04.082939 10273 master-stress-test.cc:579] Killing test thread 6/10
I20260809 01:46:04.083216 10273 master-stress-test.cc:581] Killed test thread 6/10
I20260809 01:46:04.083417 10273 master-stress-test.cc:579] Killing test thread 7/10
I20260809 01:46:04.084213 10273 master-stress-test.cc:581] Killed test thread 7/10
I20260809 01:46:04.084424 10273 master-stress-test.cc:579] Killing test thread 8/10
I20260809 01:46:04.085192 10273 master-stress-test.cc:581] Killed test thread 8/10
I20260809 01:46:04.085449 10273 master-stress-test.cc:579] Killing test thread 9/10
I20260809 01:46:04.086021 10273 master-stress-test.cc:581] Killed test thread 9/10
I20260809 01:46:04.086288 10273 master-stress-test.cc:579] Killing test thread 10/10
I20260809 01:46:04.086756 10273 master-stress-test.cc:581] Killed test thread 10/10
I20260809 01:46:04.087203 10273 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu with pid 10762
I20260809 01:46:04.097185 13377 raft_consensus.cc:493] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:04.098603 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet b93faf596f3c4dd286e600013c8db001 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:46:03 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:04.104166 13377 raft_consensus.cc:515] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.110781 11387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6c244108a4ff3a2df0585e5280294" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:46:04.115484 10588 catalog_manager.cc:5719] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67). New cstate: current_term: 1 leader_uuid: "4f60c9ac9414403d9ea5a97b103c2316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } health_report { overall_health: UNKNOWN } } }
W20260809 01:46:04.117976 11283 consensus_peers.cc:597] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:46:04.121218 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet 4071ee02730f42cea7bafe35aa4ca63d with delete_type TABLET_DATA_DELETED (replaced by tablet 9f485597f2c84b45929686f6edd856d4) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:04.127859 10598 catalog_manager.cc:5811] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349) does not have the latest schema for tablet e5af40994c204c4f82e36a53e00f2dfe (table default.Table_15c9a494eb954149a4095ebfca7505a6 [id=e995a84f2635461ca81f05d2ece827c6]). Expected version 1 got 0
I20260809 01:46:04.136453 13377 leader_election.cc:290] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:04.140698 10589 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_dfd9db20aa1a4d039db1128f13e1a6ff [id=778daa8e84c6441ab0ec13ad54453619] alter complete (version 2)
I20260809 01:46:04.141973 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet bb7a8b6d06064addb02979a7feedf756 with delete_type TABLET_DATA_DELETED (replaced by tablet 2126670a2f29408a9ba4b949f966bf76) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:04.110687 11159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93334cc766044e79d37f293bc8556fb" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
W20260809 01:46:04.138605 11062 leader_election.cc:343] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:46:04.154706 11283 leader_election.cc:343] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:04.154778 13310 log.cc:1199] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/977694496e014ef29f5d7d513e5219c0
I20260809 01:46:04.177546 13310 ts_tablet_manager.cc:1950] T 977694496e014ef29f5d7d513e5219c0 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:46:04.180917 11554 tablet_bootstrap.cc:492] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:04.181282 11554 ts_tablet_manager.cc:1403] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.513s user 0.010s sys 0.007s
I20260809 01:46:04.187552 11554 raft_consensus.cc:359] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.188155 11554 raft_consensus.cc:385] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:04.188393 11554 raft_consensus.cc:740] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:04.168421 11551 tablet_bootstrap.cc:492] T 81da8d08131d4a3ba636ecfc7a7d2951 P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:46:04.200183 11551 ts_tablet_manager.cc:1403] T 81da8d08131d4a3ba636ecfc7a7d2951 P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.468s user 0.021s sys 0.003s
I20260809 01:46:04.200512 13310 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 391) took 2509 ms. Trace:
I20260809 01:46:04.200937 13310 rpcz_store.cc:276] 0809 01:46:01.690750 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.690861 (+ 111us) service_pool.cc:224] Handling call
0809 01:46:03.721386 (+2030525us) ts_tablet_manager.cc:1151] Deleting tablet 977694496e014ef29f5d7d513e5219c0
0809 01:46:03.721440 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:03.886934 (+165494us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:04.074477 (+187543us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:04.200453 (+125976us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1972076}
I20260809 01:46:04.202363 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 977694496e014ef29f5d7d513e5219c0 (table default.Table_0908ea7070134c749de4ff154f23628e [id=148da762500547349a600027071299ab]) successfully deleted
I20260809 01:46:04.202550 11551 raft_consensus.cc:359] T 81da8d08131d4a3ba636ecfc7a7d2951 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.203166 11551 raft_consensus.cc:385] T 81da8d08131d4a3ba636ecfc7a7d2951 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:04.203509 13310 tablet_replica.cc:333] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:46:04.188962 11554 consensus_queue.cc:260] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.204512 13310 raft_consensus.cc:2243] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:46:04.205420 13310 raft_consensus.cc:2272] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:04.210455 11554 ts_tablet_manager.cc:1434] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.029s user 0.005s sys 0.001s
I20260809 01:46:04.211336 11554 tablet_bootstrap.cc:492] T 325e3d19b83545a197c2e09fc9643b5e P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:46:04.241767 10589 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_bd6eedffdf6440efa3e776b7079015d8 [id=8a2084683a7740db80e2c52d3f4c9506] alter complete (version 1)
I20260809 01:46:04.241899 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet e93334cc766044e79d37f293bc8556fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:04.258773 13310 ts_tablet_manager.cc:1916] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:46:04.252244 11062 consensus_peers.cc:597] T 1bc7dc3dfc2f4c1286ca24b8e3b20719 P 33b5652538e64ed58a09993d3188f8d7 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1bc7dc3dfc2f4c1286ca24b8e3b20719. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:46:04.203436 11551 raft_consensus.cc:740] T 81da8d08131d4a3ba636ecfc7a7d2951 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:46:04.259737 11551 consensus_queue.cc:260] T 81da8d08131d4a3ba636ecfc7a7d2951 P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.262023 11551 ts_tablet_manager.cc:1434] T 81da8d08131d4a3ba636ecfc7a7d2951 P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.062s user 0.005s sys 0.001s
I20260809 01:46:04.269719 11551 tablet_bootstrap.cc:492] T a238ad3d46054daa9435e9076ba8311d P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:46:04.287696 13161 ts_tablet_manager.cc:1929] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:46:04.288102 13161 log.cc:1199] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/68f022e6a57a4ec0a5a8c33159e97a8e
I20260809 01:46:04.288862 13161 ts_tablet_manager.cc:1950] T 68f022e6a57a4ec0a5a8c33159e97a8e P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:46:04.290828 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 402) took 2660 ms. Trace:
I20260809 01:46:04.294907 11554 tablet_bootstrap.cc:654] T 325e3d19b83545a197c2e09fc9643b5e P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:04.302707 11124 tablet_service.cc:1558] Processing DeleteTablet for tablet 81da8d08131d4a3ba636ecfc7a7d2951 with delete_type TABLET_DATA_DELETED (replaced by tablet cb13f4c7b07348cba67b6b43a9b154bd) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:04.309113 11551 tablet_bootstrap.cc:654] T a238ad3d46054daa9435e9076ba8311d P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:04.291147 13161 rpcz_store.cc:276] 0809 01:46:01.630423 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.630557 (+ 134us) service_pool.cc:224] Handling call
0809 01:46:03.809798 (+2179241us) ts_tablet_manager.cc:1151] Deleting tablet 68f022e6a57a4ec0a5a8c33159e97a8e
0809 01:46:03.809841 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:04.171508 (+361667us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:04.287567 (+116059us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:04.290777 (+ 3210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61019,"tablet-delete.queue_time_us":2178611}
I20260809 01:46:04.327885 13161 tablet_replica.cc:333] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:46:04.328125 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 68f022e6a57a4ec0a5a8c33159e97a8e (table default.Table_bbf8548d28c748e19d427c4f7f79622f [id=7e63b2e8084c4dc9b53d011ba1d822e6]) successfully deleted
I20260809 01:46:04.328639 13161 raft_consensus.cc:2243] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:04.329099 13161 raft_consensus.cc:2272] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260809 01:46:04.332446 13262 log.cc:927] Time spent T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88: Append to log took a long time: real 0.056s user 0.005s sys 0.005s
I20260809 01:46:04.346855 13161 ts_tablet_manager.cc:1916] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:04.351918 11554 tablet_bootstrap.cc:492] T 325e3d19b83545a197c2e09fc9643b5e P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:04.352340 11554 ts_tablet_manager.cc:1403] T 325e3d19b83545a197c2e09fc9643b5e P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.141s user 0.012s sys 0.000s
I20260809 01:46:04.352731 13310 ts_tablet_manager.cc:1929] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:46:04.353240 13310 log.cc:1199] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/68f022e6a57a4ec0a5a8c33159e97a8e
I20260809 01:46:04.354097 13310 ts_tablet_manager.cc:1950] T 68f022e6a57a4ec0a5a8c33159e97a8e P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:46:04.354890 11554 raft_consensus.cc:359] T 325e3d19b83545a197c2e09fc9643b5e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:46:04.355644 11554 raft_consensus.cc:385] T 325e3d19b83545a197c2e09fc9643b5e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:04.355974 11554 raft_consensus.cc:740] T 325e3d19b83545a197c2e09fc9643b5e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:04.372686 11554 consensus_queue.cc:260] T 325e3d19b83545a197c2e09fc9643b5e P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } } peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } }
I20260809 01:46:04.402323 11554 ts_tablet_manager.cc:1434] T 325e3d19b83545a197c2e09fc9643b5e P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.050s user 0.007s sys 0.000s
I20260809 01:46:04.426635 13310 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 394) took 2671 ms. Trace:
I20260809 01:46:04.427467 13310 rpcz_store.cc:276] 0809 01:46:01.754837 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.755110 (+ 273us) service_pool.cc:224] Handling call
0809 01:46:04.203288 (+2448178us) ts_tablet_manager.cc:1151] Deleting tablet 68f022e6a57a4ec0a5a8c33159e97a8e
0809 01:46:04.203331 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:04.332200 (+128869us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:04.352603 (+ 20403us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:04.426331 (+ 73728us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2447546}
I20260809 01:46:04.429169 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 68f022e6a57a4ec0a5a8c33159e97a8e (table default.Table_bbf8548d28c748e19d427c4f7f79622f [id=7e63b2e8084c4dc9b53d011ba1d822e6]) successfully deleted
I20260809 01:46:04.429932 13310 tablet_replica.cc:333] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:46:04.434926 13395 tablet.cc:1724] T 3e03033672e947eab8cc143263281133 P 33b5652538e64ed58a09993d3188f8d7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260809 01:46:04.442734 11062 consensus_peers.cc:597] T 33b297f9083744e2953d41cd7b1055df P 33b5652538e64ed58a09993d3188f8d7 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260809 01:46:04.446723 13418 tablet.cc:1724] T 3e03033672e947eab8cc143263281133 P 4f60c9ac9414403d9ea5a97b103c2316: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 01:46:04.458808 11554 tablet_bootstrap.cc:492] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:46:04.510818 13161 ts_tablet_manager.cc:1929] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:46:04.522987 13161 log.cc:1199] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/a38bd344108649ed8e1687f82154000b
I20260809 01:46:04.529956 13161 ts_tablet_manager.cc:1950] T a38bd344108649ed8e1687f82154000b P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:46:04.543879 11551 tablet_bootstrap.cc:492] T a238ad3d46054daa9435e9076ba8311d P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:46:04.544342 11551 ts_tablet_manager.cc:1403] T a238ad3d46054daa9435e9076ba8311d P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.275s user 0.011s sys 0.004s
I20260809 01:46:04.534565 11554 tablet_bootstrap.cc:654] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:04.550140 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 404) took 2890 ms. Trace:
I20260809 01:46:04.550595 13161 rpcz_store.cc:276] 0809 01:46:01.659492 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.659618 (+ 126us) service_pool.cc:224] Handling call
0809 01:46:04.327719 (+2668101us) ts_tablet_manager.cc:1151] Deleting tablet a38bd344108649ed8e1687f82154000b
0809 01:46:04.327762 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:04.405366 (+ 77604us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:04.510700 (+105334us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:04.550069 (+ 39369us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2667552}
I20260809 01:46:04.552137 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet a38bd344108649ed8e1687f82154000b (table default.Table_834b6f679f5a401c9a32de707d5b931d [id=4c630a680e3f41c9a92e50872d245039]) successfully deleted
I20260809 01:46:04.553236 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 403) took 2894 ms. Trace:
I20260809 01:46:04.557874 11551 raft_consensus.cc:359] T a238ad3d46054daa9435e9076ba8311d P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.558741 11551 raft_consensus.cc:385] T a238ad3d46054daa9435e9076ba8311d P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:04.559001 11551 raft_consensus.cc:740] T a238ad3d46054daa9435e9076ba8311d P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:46:04.564723 10588 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59: default.Table_109c891eea264584b9e1ceab647f32aa [id=24d2e675d969421fba4647380880f606] alter complete (version 3)
I20260809 01:46:04.563941 13310 raft_consensus.cc:2243] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:04.572870 13310 raft_consensus.cc:2272] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:04.575706 13310 ts_tablet_manager.cc:1916] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:04.559629 11551 consensus_queue.cc:260] T a238ad3d46054daa9435e9076ba8311d P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.581617 13161 rpcz_store.cc:276] 0809 01:46:01.659203 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.659636 (+ 433us) service_pool.cc:224] Handling call
0809 01:46:04.552994 (+2893358us) ts_tablet_manager.cc:1151] Deleting tablet b9493892c1ff48e7a0558ff0fcb7a0f3
0809 01:46:04.553034 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:04.553184 (+ 150us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2892308}
I20260809 01:46:04.587944 13161 tablet_replica.cc:333] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:46:04.594847 11551 ts_tablet_manager.cc:1434] T a238ad3d46054daa9435e9076ba8311d P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.050s user 0.006s sys 0.001s
I20260809 01:46:04.595999 11551 tablet_bootstrap.cc:492] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:46:04.594686 13161 raft_consensus.cc:2243] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
W20260809 01:46:04.585721 10579 catalog_manager.cc:5017] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet b9493892c1ff48e7a0558ff0fcb7a0f3 because the tablet was not found. No further retry: Not found: Tablet not found: b9493892c1ff48e7a0558ff0fcb7a0f3
I20260809 01:46:04.602006 13334 raft_consensus.cc:493] T 67b31998b65e4264b16c0d73dbcfc73b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:04.602495 13334 raft_consensus.cc:515] T 67b31998b65e4264b16c0d73dbcfc73b P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.606171 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b31998b65e4264b16c0d73dbcfc73b" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:04.607694 11062 leader_election.cc:343] T 67b31998b65e4264b16c0d73dbcfc73b P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:04.608579 13161 raft_consensus.cc:2272] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:04.609207 13334 leader_election.cc:290] T 67b31998b65e4264b16c0d73dbcfc73b P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:04.610466 13161 ts_tablet_manager.cc:1916] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:04.631166 11554 tablet_bootstrap.cc:492] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:04.631647 11554 ts_tablet_manager.cc:1403] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.173s user 0.010s sys 0.000s
I20260809 01:46:04.652676 11551 tablet_bootstrap.cc:654] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:04.667390 11554 raft_consensus.cc:359] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.668022 11554 raft_consensus.cc:385] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:04.668253 11554 raft_consensus.cc:740] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:04.668826 11554 consensus_queue.cc:260] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.676183 11554 ts_tablet_manager.cc:1434] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.044s user 0.004s sys 0.000s
I20260809 01:46:04.677275 11395 raft_consensus.cc:3060] T 325e3d19b83545a197c2e09fc9643b5e P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:04.702155 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet c3bf43978dc94c2b9a7f3c18a4677c40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:43 UTC) from {username='slave'} at 127.0.0.1:53476
I20260809 01:46:04.706283 11554 tablet_bootstrap.cc:492] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:46:04.741037 11554 tablet_bootstrap.cc:654] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
W20260809 01:46:04.748032 11062 consensus_peers.cc:597] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 33b5652538e64ed58a09993d3188f8d7 -> Peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Couldn't send request to peer 4f60c9ac9414403d9ea5a97b103c2316. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260809 01:46:04.730870 13310 ts_tablet_manager.cc:1929] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:46:04.749372 13310 log.cc:1199] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/761370ec8b164fd184a2e091030f610b
I20260809 01:46:04.750579 13310 ts_tablet_manager.cc:1950] T 761370ec8b164fd184a2e091030f610b P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:46:04.742583 13161 ts_tablet_manager.cc:1929] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:46:04.756739 13161 log.cc:1199] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/cd09bea5263344919ce26593fe958547
I20260809 01:46:04.757622 13161 ts_tablet_manager.cc:1950] T cd09bea5263344919ce26593fe958547 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:46:04.759718 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 405) took 3095 ms. Trace:
I20260809 01:46:04.765607 13161 rpcz_store.cc:276] 0809 01:46:01.664116 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.664227 (+ 111us) service_pool.cc:224] Handling call
0809 01:46:04.587078 (+2922851us) ts_tablet_manager.cc:1151] Deleting tablet cd09bea5263344919ce26593fe958547
0809 01:46:04.587302 (+ 224us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:04.669791 (+ 82489us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:04.742469 (+ 72678us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:04.759652 (+ 17183us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2922008}
I20260809 01:46:04.766937 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet cd09bea5263344919ce26593fe958547 (table default.Table_6f6560d974c74f2dbfe00a79fc671fcb [id=cb35a16a1e1241abb696b413e3c1d8cd]) successfully deleted
I20260809 01:46:04.770015 13377 raft_consensus.cc:493] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:04.770417 13377 raft_consensus.cc:515] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.772452 13377 leader_election.cc:290] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:04.777022 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 406) took 2953 ms. Trace:
I20260809 01:46:04.777343 13161 rpcz_store.cc:276] 0809 01:46:01.823886 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.823985 (+ 99us) service_pool.cc:224] Handling call
0809 01:46:04.767646 (+2943661us) ts_tablet_manager.cc:1151] Deleting tablet e0fdf4f4b1d44979936b37886bc60836
0809 01:46:04.776656 (+ 9010us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:04.776977 (+ 321us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2935073}
I20260809 01:46:04.778473 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 411) took 2717 ms. Trace:
I20260809 01:46:04.778743 13161 rpcz_store.cc:276] 0809 01:46:02.060570 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:02.060686 (+ 116us) service_pool.cc:224] Handling call
0809 01:46:04.778158 (+2717472us) ts_tablet_manager.cc:1151] Deleting tablet 5e9afec0618e4fa293f6dad6769ae49c
0809 01:46:04.778193 (+ 35us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:04.778427 (+ 234us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2716949}
I20260809 01:46:04.779754 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 410) took 2719 ms. Trace:
I20260809 01:46:04.780009 13161 rpcz_store.cc:276] 0809 01:46:02.060264 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:02.061220 (+ 956us) service_pool.cc:224] Handling call
0809 01:46:04.779500 (+2718280us) ts_tablet_manager.cc:1151] Deleting tablet acc9c8e0f97e41b48a919aa579ad85cd
0809 01:46:04.779533 (+ 33us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:04.779714 (+ 181us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2717722}
I20260809 01:46:04.780861 13161 tablet_replica.cc:333] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:46:04.772940 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6c244108a4ff3a2df0585e5280294" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:04.781951 11158 raft_consensus.cc:2393] T afb6c244108a4ff3a2df0585e5280294 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f60c9ac9414403d9ea5a97b103c2316 in current term 1: Already voted for candidate 33b5652538e64ed58a09993d3188f8d7 in this term.
W20260809 01:46:04.786067 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet e0fdf4f4b1d44979936b37886bc60836 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e0fdf4f4b1d44979936b37886bc60836 already in progress: creating tablet
W20260809 01:46:04.786705 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 5e9afec0618e4fa293f6dad6769ae49c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5e9afec0618e4fa293f6dad6769ae49c already in progress: creating tablet
W20260809 01:46:04.787269 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet acc9c8e0f97e41b48a919aa579ad85cd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet acc9c8e0f97e41b48a919aa579ad85cd already in progress: creating tablet
I20260809 01:46:04.790364 13310 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 393) took 3038 ms. Trace:
I20260809 01:46:04.790710 13310 rpcz_store.cc:276] 0809 01:46:01.751516 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.751616 (+ 100us) service_pool.cc:224] Handling call
0809 01:46:04.429655 (+2678039us) ts_tablet_manager.cc:1151] Deleting tablet 761370ec8b164fd184a2e091030f610b
0809 01:46:04.429771 (+ 116us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:04.657755 (+227984us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:04.730729 (+ 72974us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:04.790227 (+ 59498us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":132680,"tablet-delete.queue_time_us":2661258}
I20260809 01:46:04.792177 13310 tablet_replica.cc:333] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:46:04.793176 13310 raft_consensus.cc:2243] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:04.793334 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 761370ec8b164fd184a2e091030f610b (table default.Table_4d3af03d62f44239afda0e4122e17b2d [id=3e8bf3dc52ab433b84eeafb2c2685e21]) successfully deleted
I20260809 01:46:04.795212 11551 tablet_bootstrap.cc:492] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:46:04.795612 11551 ts_tablet_manager.cc:1403] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.200s user 0.011s sys 0.002s
I20260809 01:46:04.802166 13161 raft_consensus.cc:2243] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:46:04.805405 11554 tablet_bootstrap.cc:492] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:04.805838 11554 ts_tablet_manager.cc:1403] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.100s user 0.009s sys 0.000s
I20260809 01:46:04.808357 11554 raft_consensus.cc:359] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.808954 11554 raft_consensus.cc:385] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:04.809198 11554 raft_consensus.cc:740] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:04.802910 13161 raft_consensus.cc:2272] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:04.811085 13161 ts_tablet_manager.cc:1916] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:04.838429 13310 raft_consensus.cc:2272] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:04.840226 13334 raft_consensus.cc:493] T 81da8d08131d4a3ba636ecfc7a7d2951 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:04.840703 13334 raft_consensus.cc:515] T 81da8d08131d4a3ba636ecfc7a7d2951 P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.841436 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 80b4c9c4b9fb4a6e8b767c7206233ee1 (DEFAULT_TABLE table=default.Table_267d92fa9322471a9e85b9bf80061ff3 [id=685f490d9f254c6da970f102d409b00c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:04.842485 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80b4c9c4b9fb4a6e8b767c7206233ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:04.842679 13334 leader_election.cc:290] T 81da8d08131d4a3ba636ecfc7a7d2951 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:04.844192 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81da8d08131d4a3ba636ecfc7a7d2951" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:04.846673 11062 leader_election.cc:343] T 81da8d08131d4a3ba636ecfc7a7d2951 P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
I20260809 01:46:04.848090 11551 raft_consensus.cc:359] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.848752 11551 raft_consensus.cc:385] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:04.850067 11551 raft_consensus.cc:740] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:46:04.850834 11551 consensus_queue.cc:260] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.853926 11554 consensus_queue.cc:260] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:04.856143 11554 ts_tablet_manager.cc:1434] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.050s user 0.005s sys 0.000s
I20260809 01:46:04.862452 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 80b4c9c4b9fb4a6e8b767c7206233ee1 (DEFAULT_TABLE table=default.Table_267d92fa9322471a9e85b9bf80061ff3 [id=685f490d9f254c6da970f102d409b00c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:04.863590 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80b4c9c4b9fb4a6e8b767c7206233ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:04.865989 11336 tablet_service.cc:1511] Processing CreateTablet for tablet ee6dbf65ab424ba493f98cce6ca60672 (DEFAULT_TABLE table=default.Table_23821a5167f24f9882d9b3dc6edd65bb [id=615e02f78b2d4a158db47e11d68ae20f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:04.880000 11106 tablet_service.cc:1511] Processing CreateTablet for tablet ee6dbf65ab424ba493f98cce6ca60672 (DEFAULT_TABLE table=default.Table_23821a5167f24f9882d9b3dc6edd65bb [id=615e02f78b2d4a158db47e11d68ae20f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:04.882143 11551 ts_tablet_manager.cc:1434] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.086s user 0.005s sys 0.003s
I20260809 01:46:04.882426 11554 tablet_bootstrap.cc:492] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:46:04.885720 11349 tablet_service.cc:1511] Processing CreateTablet for tablet 0e84da6d080d4617bc637488991bfec7 (DEFAULT_TABLE table=default.Table_8261d1a2fbb241ac8e38755611662bad [id=086d5f9c38b44f45b819c56658251abf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:04.886157 11106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee6dbf65ab424ba493f98cce6ca60672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:04.889358 11114 tablet_service.cc:1511] Processing CreateTablet for tablet 0e84da6d080d4617bc637488991bfec7 (DEFAULT_TABLE table=default.Table_8261d1a2fbb241ac8e38755611662bad [id=086d5f9c38b44f45b819c56658251abf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:04.890511 11114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e84da6d080d4617bc637488991bfec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:04.891121 11554 tablet_bootstrap.cc:654] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:04.894384 11551 tablet_bootstrap.cc:492] T acc9c8e0f97e41b48a919aa579ad85cd P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:46:04.895656 13310 ts_tablet_manager.cc:1916] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:04.902765 11336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee6dbf65ab424ba493f98cce6ca60672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:04.906010 11349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e84da6d080d4617bc637488991bfec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:04.931236 11361 tablet_service.cc:1558] Processing DeleteTablet for tablet 50f45c02a6994c5aaaf9c0a499e15fed with delete_type TABLET_DATA_DELETED (replaced by tablet ebc209c696ac4c669961f5ed8392af40) from {username='slave'} at 127.0.0.1:53476
W20260809 01:46:04.936707 10576 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 12fb852b8ea8478cb160a265459865c4 (table default.Table_996edb36352849a59f4ecb3388abaf8a [id=212edfae1bf849c3b46674ec191ee76d], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 12fb852b8ea8478cb160a265459865c4 consensus state has no leader
I20260809 01:46:04.949680 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 51fdf2aad6294ff4be9f0614081871df (DEFAULT_TABLE table=default.Table_c09873b499f949898bd861175f2c9c6b [id=147368855c2c46f18fa928288dd78f46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:04.950546 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51fdf2aad6294ff4be9f0614081871df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:04.952767 11124 tablet_service.cc:1511] Processing CreateTablet for tablet 51fdf2aad6294ff4be9f0614081871df (DEFAULT_TABLE table=default.Table_c09873b499f949898bd861175f2c9c6b [id=147368855c2c46f18fa928288dd78f46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:04.954599 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51fdf2aad6294ff4be9f0614081871df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:04.963080 11107 tablet_service.cc:1558] Processing DeleteTablet for tablet e93334cc766044e79d37f293bc8556fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:45:46 UTC) from {username='slave'} at 127.0.0.1:56566
I20260809 01:46:04.970240 11551 tablet_bootstrap.cc:654] T acc9c8e0f97e41b48a919aa579ad85cd P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:04.975421 13161 ts_tablet_manager.cc:1929] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:46:04.987546 13310 ts_tablet_manager.cc:1929] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:46:05.029261 11107 tablet_service.cc:1511] Processing CreateTablet for tablet c69e03138427404eac6727478a93d6af (DEFAULT_TABLE table=default.Table_c513b1a1e5f84adcbda1576873b3ba41 [id=010db3c3b8104e2e827950dfd9eca9f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.029132 13310 log.cc:1199] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/bb7a8b6d06064addb02979a7feedf756
I20260809 01:46:05.030360 11107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69e03138427404eac6727478a93d6af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.032320 13310 ts_tablet_manager.cc:1950] T bb7a8b6d06064addb02979a7feedf756 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:46:05.034492 11119 tablet_service.cc:1511] Processing CreateTablet for tablet 425d5d58df494173a31a1c90e1ffed47 (DEFAULT_TABLE table=default.Table_cdc79f33d04b4f6db40aa0dd540b8b4d [id=e1341f5b3632434d823316380e754cf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.035544 11119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 425d5d58df494173a31a1c90e1ffed47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.037751 11126 tablet_service.cc:1511] Processing CreateTablet for tablet f6542267bb2b4c4eb3800e719536167e (DEFAULT_TABLE table=default.Table_a7e3291583454a0a83b95fa7312eb2b7 [id=c2f4c7dff5324833a6ea6666519ac4c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.038316 13310 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 396) took 3221 ms. Trace:
I20260809 01:46:05.038729 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6542267bb2b4c4eb3800e719536167e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.038671 13310 rpcz_store.cc:276] 0809 01:46:01.817377 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.825643 (+ 8266us) service_pool.cc:224] Handling call
0809 01:46:04.791995 (+2966352us) ts_tablet_manager.cc:1151] Deleting tablet bb7a8b6d06064addb02979a7feedf756
0809 01:46:04.792048 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:04.967279 (+175231us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:04.987428 (+ 20149us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:05.038249 (+ 50821us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54783,"tablet-delete.queue_time_us":2963162}
I20260809 01:46:04.993009 13377 raft_consensus.cc:493] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:05.048364 13377 raft_consensus.cc:515] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.050331 13377 leader_election.cc:290] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:04.992472 11336 tablet_service.cc:1511] Processing CreateTablet for tablet c69e03138427404eac6727478a93d6af (DEFAULT_TABLE table=default.Table_c513b1a1e5f84adcbda1576873b3ba41 [id=010db3c3b8104e2e827950dfd9eca9f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.052469 11336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69e03138427404eac6727478a93d6af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.003947 13431 raft_consensus.cc:493] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 01:46:05.057931 11283 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260809 01:46:05.057731 13431 raft_consensus.cc:515] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.059501 13431 leader_election.cc:290] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:05.010108 11349 tablet_service.cc:1511] Processing CreateTablet for tablet 425d5d58df494173a31a1c90e1ffed47 (DEFAULT_TABLE table=default.Table_cdc79f33d04b4f6db40aa0dd540b8b4d [id=e1341f5b3632434d823316380e754cf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.060418 11349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 425d5d58df494173a31a1c90e1ffed47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.020853 11351 tablet_service.cc:1511] Processing CreateTablet for tablet f6542267bb2b4c4eb3800e719536167e (DEFAULT_TABLE table=default.Table_a7e3291583454a0a83b95fa7312eb2b7 [id=c2f4c7dff5324833a6ea6666519ac4c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.066183 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6542267bb2b4c4eb3800e719536167e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.066677 11341 tablet_service.cc:1511] Processing CreateTablet for tablet e04e905d1a7c4d959ffb8124346fd55a (DEFAULT_TABLE table=default.Table_bb09705b708949a8a4600ab12d1d7704 [id=f5f27ecbf2a04ad7946e4bd383a9f3d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.067581 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e04e905d1a7c4d959ffb8124346fd55a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.083055 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 30fd4c194e224c05ba9a0a183c48d260 (DEFAULT_TABLE table=default.Table_83678aa7b12042f1b89ee51be4cb1d7a [id=1b1305e03eec4d11b621c620d4a63676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.085924 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30fd4c194e224c05ba9a0a183c48d260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.087949 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f45c02a6994c5aaaf9c0a499e15fed" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7" is_pre_election: true
I20260809 01:46:05.088474 11158 raft_consensus.cc:2468] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 0.
I20260809 01:46:05.090656 13310 tablet_replica.cc:333] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:46:05.091492 13310 raft_consensus.cc:2243] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:05.091982 13310 raft_consensus.cc:2272] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:05.094023 11116 tablet_service.cc:1511] Processing CreateTablet for tablet e04e905d1a7c4d959ffb8124346fd55a (DEFAULT_TABLE table=default.Table_bb09705b708949a8a4600ab12d1d7704 [id=f5f27ecbf2a04ad7946e4bd383a9f3d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.094828 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 30fd4c194e224c05ba9a0a183c48d260 (DEFAULT_TABLE table=default.Table_83678aa7b12042f1b89ee51be4cb1d7a [id=1b1305e03eec4d11b621c620d4a63676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.095111 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e04e905d1a7c4d959ffb8124346fd55a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.095851 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30fd4c194e224c05ba9a0a183c48d260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.098076 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet bb7a8b6d06064addb02979a7feedf756 (table default.Table_7399370c91a7495faa66e98683c688d0 [id=43255c746b5c4fcc91454fcad2c801a6]) successfully deleted
I20260809 01:46:05.099010 11283 leader_election.cc:304] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:46:05.100529 13431 raft_consensus.cc:2804] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:05.100837 13431 raft_consensus.cc:493] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:05.101161 13431 raft_consensus.cc:3060] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:46:05.105156 11283 consensus_peers.cc:597] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb7a8b6d06064addb02979a7feedf756. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:46:05.106904 11346 tablet_service.cc:1511] Processing CreateTablet for tablet 1ac86fb23ad542329f1208a2defd9c06 (DEFAULT_TABLE table=default.Table_5415a483bead450ead7429146a04fb93 [id=57e3c2cdc1834e3799bad387da1d61b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.106210 13423 raft_consensus.cc:493] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:05.107964 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ac86fb23ad542329f1208a2defd9c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.107695 13423 raft_consensus.cc:515] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.109623 13423 leader_election.cc:290] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:05.048381 13161 log.cc:1199] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/761370ec8b164fd184a2e091030f610b
I20260809 01:46:05.118382 13161 ts_tablet_manager.cc:1950] T 761370ec8b164fd184a2e091030f610b P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:46:05.118813 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93334cc766044e79d37f293bc8556fb" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
I20260809 01:46:05.119316 11395 raft_consensus.cc:2468] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 0.
I20260809 01:46:05.120244 13431 raft_consensus.cc:515] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.121461 11554 tablet_bootstrap.cc:492] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:05.121881 11554 ts_tablet_manager.cc:1403] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.240s user 0.019s sys 0.025s
I20260809 01:46:05.123271 11345 tablet_service.cc:1511] Processing CreateTablet for tablet fd957107e602439dbef9d7d8dbfe412f (DEFAULT_TABLE table=default.Table_487a0ef0bc0a44cd9ca61c752857b15e [id=f5381e78378d4e99bc7be62fbceef5c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.124332 11345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd957107e602439dbef9d7d8dbfe412f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.124243 11554 raft_consensus.cc:359] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.124778 11554 raft_consensus.cc:385] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:05.125018 11554 raft_consensus.cc:740] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:05.120471 11062 leader_election.cc:304] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:46:05.130614 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 06b7cdf99c9042fd8d7779a4e33a3c8f (DEFAULT_TABLE table=default.Table_5a885f5331e5438a9dd35554f82d3d92 [id=ed814c88bb7f471b991d0d8fbc777390]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.131475 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b7cdf99c9042fd8d7779a4e33a3c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.131989 11111 tablet_service.cc:1511] Processing CreateTablet for tablet 06b7cdf99c9042fd8d7779a4e33a3c8f (DEFAULT_TABLE table=default.Table_5a885f5331e5438a9dd35554f82d3d92 [id=ed814c88bb7f471b991d0d8fbc777390]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.133054 11111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b7cdf99c9042fd8d7779a4e33a3c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.133953 11124 tablet_service.cc:1511] Processing CreateTablet for tablet fd957107e602439dbef9d7d8dbfe412f (DEFAULT_TABLE table=default.Table_487a0ef0bc0a44cd9ca61c752857b15e [id=f5381e78378d4e99bc7be62fbceef5c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.134816 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd957107e602439dbef9d7d8dbfe412f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.131968 11105 tablet_service.cc:1511] Processing CreateTablet for tablet 1ac86fb23ad542329f1208a2defd9c06 (DEFAULT_TABLE table=default.Table_5415a483bead450ead7429146a04fb93 [id=57e3c2cdc1834e3799bad387da1d61b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.136566 11105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ac86fb23ad542329f1208a2defd9c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.137717 13423 raft_consensus.cc:2804] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:05.138044 13423 raft_consensus.cc:493] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:05.138335 13423 raft_consensus.cc:3060] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:46:05.141961 11283 consensus_peers.cc:597] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Couldn't send request to peer 33b5652538e64ed58a09993d3188f8d7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
W20260809 01:46:05.142513 11283 leader_election.cc:343] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Not found: Tablet not found: c3bf43978dc94c2b9a7f3c18a4677c40
I20260809 01:46:05.144968 13310 ts_tablet_manager.cc:1916] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:05.145757 11347 tablet_service.cc:1511] Processing CreateTablet for tablet d025221e348c427da00fb7c71a3b211a (DEFAULT_TABLE table=default.Table_77debdadf8e346f388aac796f7d2cd09 [id=58c20c656db54d46aa66f2baf0f7412a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.146651 11347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d025221e348c427da00fb7c71a3b211a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.146839 11120 tablet_service.cc:1511] Processing CreateTablet for tablet d025221e348c427da00fb7c71a3b211a (DEFAULT_TABLE table=default.Table_77debdadf8e346f388aac796f7d2cd09 [id=58c20c656db54d46aa66f2baf0f7412a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.147807 11120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d025221e348c427da00fb7c71a3b211a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.162360 11334 tablet_service.cc:1511] Processing CreateTablet for tablet 448f81040df24c569304de21774ea867 (DEFAULT_TABLE table=default.Table_ae6c8e43efc446c2887d20c55c3f6fe3 [id=6f0fe9223eab48e78a65d30f009ef913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.163328 11334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 448f81040df24c569304de21774ea867. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.166632 11158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f45c02a6994c5aaaf9c0a499e15fed" candidate_uuid: "4f60c9ac9414403d9ea5a97b103c2316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b5652538e64ed58a09993d3188f8d7"
I20260809 01:46:05.167145 11158 raft_consensus.cc:3060] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:05.168995 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 5b944041b1494ddc9e2ffe6edba0f776 (DEFAULT_TABLE table=default.Table_050ffc9040064a909b37534a21ce9876 [id=cb42f672dd2d4310b56978b2678c434f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.169842 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b944041b1494ddc9e2ffe6edba0f776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.176313 13423 raft_consensus.cc:515] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.178709 11158 raft_consensus.cc:2468] T 50f45c02a6994c5aaaf9c0a499e15fed P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f60c9ac9414403d9ea5a97b103c2316 in term 1.
I20260809 01:46:05.189638 13431 leader_election.cc:290] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Requested vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:05.181731 11117 tablet_service.cc:1511] Processing CreateTablet for tablet 5b944041b1494ddc9e2ffe6edba0f776 (DEFAULT_TABLE table=default.Table_050ffc9040064a909b37534a21ce9876 [id=cb42f672dd2d4310b56978b2678c434f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.120266 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 412) took 3059 ms. Trace:
I20260809 01:46:05.198509 11117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b944041b1494ddc9e2ffe6edba0f776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.198104 13161 rpcz_store.cc:276] 0809 01:46:02.060844 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:02.061323 (+ 479us) service_pool.cc:224] Handling call
0809 01:46:04.780744 (+2719421us) ts_tablet_manager.cc:1151] Deleting tablet 761370ec8b164fd184a2e091030f610b
0809 01:46:04.780778 (+ 34us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:04.911097 (+130319us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:04.975310 (+ 64213us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:05.120213 (+144903us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2718845}
I20260809 01:46:05.199257 13161 tablet_replica.cc:333] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
I20260809 01:46:05.199476 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 761370ec8b164fd184a2e091030f610b (table default.Table_4d3af03d62f44239afda0e4122e17b2d [id=3e8bf3dc52ab433b84eeafb2c2685e21]) successfully deleted
I20260809 01:46:05.199954 13161 raft_consensus.cc:2243] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:46:05.200687 13161 raft_consensus.cc:2272] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:05.206064 11341 tablet_service.cc:1511] Processing CreateTablet for tablet bb1021604a8149c08dad415f6f9643e9 (DEFAULT_TABLE table=default.Table_95568eee8c064b09bcb94444d7839a77 [id=7cf798660a40497fbb9aea91a567e194]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.207046 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb1021604a8149c08dad415f6f9643e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.209654 11114 tablet_service.cc:1511] Processing CreateTablet for tablet bb1021604a8149c08dad415f6f9643e9 (DEFAULT_TABLE table=default.Table_95568eee8c064b09bcb94444d7839a77 [id=7cf798660a40497fbb9aea91a567e194]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.210858 11283 leader_election.cc:304] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:46:05.212864 13431 raft_consensus.cc:2804] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:05.214511 13431 raft_consensus.cc:697] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 LEADER]: Becoming Leader. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Running, Role: LEADER
I20260809 01:46:05.182229 11106 tablet_service.cc:1511] Processing CreateTablet for tablet 448f81040df24c569304de21774ea867 (DEFAULT_TABLE table=default.Table_ae6c8e43efc446c2887d20c55c3f6fe3 [id=6f0fe9223eab48e78a65d30f009ef913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.216065 13431 consensus_queue.cc:237] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.217161 11123 tablet_service.cc:1511] Processing CreateTablet for tablet 0df890d23ccd4b698981f32d80a9bb9d (DEFAULT_TABLE table=default.Table_2142392a3c3b4314863176189fc985d4 [id=5aa2fa4187b9472cba33812fb5ccb51b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.218521 11114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb1021604a8149c08dad415f6f9643e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.220892 11123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df890d23ccd4b698981f32d80a9bb9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.226874 13470 raft_consensus.cc:493] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 629f002048f64491afdd57f72a466e59)
I20260809 01:46:05.227411 13470 raft_consensus.cc:515] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } }
I20260809 01:46:05.229828 11106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 448f81040df24c569304de21774ea867. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.230127 11366 tablet_service.cc:1511] Processing CreateTablet for tablet 0df890d23ccd4b698981f32d80a9bb9d (DEFAULT_TABLE table=default.Table_2142392a3c3b4314863176189fc985d4 [id=5aa2fa4187b9472cba33812fb5ccb51b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.230993 11366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df890d23ccd4b698981f32d80a9bb9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.233768 11554 consensus_queue.cc:260] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.235953 11554 ts_tablet_manager.cc:1434] T 63c1a610663a4a2fa58f6cdc5f8d86ce P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.114s user 0.004s sys 0.001s
I20260809 01:46:05.236879 11554 tablet_bootstrap.cc:492] T 82d5a709cd86476594613e788a325094 P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
W20260809 01:46:05.237612 11283 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260809 01:46:05.238735 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93334cc766044e79d37f293bc8556fb" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316"
I20260809 01:46:05.239238 11395 raft_consensus.cc:3060] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:05.246129 11395 raft_consensus.cc:2468] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b5652538e64ed58a09993d3188f8d7 in term 1.
I20260809 01:46:05.247426 11062 leader_election.cc:304] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b5652538e64ed58a09993d3188f8d7, 4f60c9ac9414403d9ea5a97b103c2316; no voters:
I20260809 01:46:05.248178 13334 raft_consensus.cc:2804] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:05.250358 11349 tablet_service.cc:1511] Processing CreateTablet for tablet c2a80d5455674b559597fbcc770ec635 (DEFAULT_TABLE table=default.Table_a877cd3e3d1846b597c39578812fbdcf [id=38c09da48d9c49c794c386da27c22f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.251863 11349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2a80d5455674b559597fbcc770ec635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.254470 11347 tablet_service.cc:1511] Processing CreateTablet for tablet 6d83d6c1acf34440a37b9d2572a216da (DEFAULT_TABLE table=default.Table_9bcc86385a0840debdb02e97c0bd2144 [id=a96db20d48304ba6ad6413d78774c4b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.255618 11347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d83d6c1acf34440a37b9d2572a216da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.256757 13161 ts_tablet_manager.cc:1916] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:05.259346 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 453 } } ignore_live_leader: false dest_uuid: "09a73ee8e8ec496a984efcab27715506" is_pre_election: true
I20260809 01:46:05.259533 10612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71bc72deb1d2402cabd817dfe3b27e88" candidate_term: 3 candidate_status { last_received { term: 2 index: 453 } } ignore_live_leader: false dest_uuid: "629f002048f64491afdd57f72a466e59" is_pre_election: true
I20260809 01:46:05.261040 12870 leader_election.cc:304] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bc72deb1d2402cabd817dfe3b27e88; no voters: 09a73ee8e8ec496a984efcab27715506, 629f002048f64491afdd57f72a466e59
I20260809 01:46:05.265623 11119 tablet_service.cc:1511] Processing CreateTablet for tablet c2a80d5455674b559597fbcc770ec635 (DEFAULT_TABLE table=default.Table_a877cd3e3d1846b597c39578812fbdcf [id=38c09da48d9c49c794c386da27c22f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.266651 11119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2a80d5455674b559597fbcc770ec635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.267786 13423 leader_election.cc:290] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:05.268167 13470 leader_election.cc:290] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157), 629f002048f64491afdd57f72a466e59 (127.10.8.124:33929)
I20260809 01:46:05.269312 13476 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 71bc72deb1d2402cabd817dfe3b27e88 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 01:46:05.273379 11551 tablet_bootstrap.cc:492] T acc9c8e0f97e41b48a919aa579ad85cd P 33b5652538e64ed58a09993d3188f8d7: No bootstrap required, opened a new log
I20260809 01:46:05.273758 11551 ts_tablet_manager.cc:1403] T acc9c8e0f97e41b48a919aa579ad85cd P 33b5652538e64ed58a09993d3188f8d7: Time spent bootstrapping tablet: real 0.380s user 0.013s sys 0.009s
I20260809 01:46:05.275974 11551 raft_consensus.cc:359] T acc9c8e0f97e41b48a919aa579ad85cd P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.276561 11551 raft_consensus.cc:385] T acc9c8e0f97e41b48a919aa579ad85cd P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:05.281606 13334 raft_consensus.cc:697] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [term 1 LEADER]: Becoming Leader. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Running, Role: LEADER
I20260809 01:46:05.284797 13334 consensus_queue.cc:237] T e93334cc766044e79d37f293bc8556fb P 33b5652538e64ed58a09993d3188f8d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.289235 11111 tablet_service.cc:1511] Processing CreateTablet for tablet 65fa053480ee446685796edc48772aa5 (DEFAULT_TABLE table=default.Table_8dde73a0e7f44af28232bbb7a0d57897 [id=cf06c89e274a4b81b578c53aaf7cd80a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.290189 11111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65fa053480ee446685796edc48772aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.294447 11346 tablet_service.cc:1511] Processing CreateTablet for tablet 65fa053480ee446685796edc48772aa5 (DEFAULT_TABLE table=default.Table_8dde73a0e7f44af28232bbb7a0d57897 [id=cf06c89e274a4b81b578c53aaf7cd80a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.299057 11346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65fa053480ee446685796edc48772aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.276798 11551 raft_consensus.cc:740] T acc9c8e0f97e41b48a919aa579ad85cd P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b5652538e64ed58a09993d3188f8d7, State: Initialized, Role: FOLLOWER
I20260809 01:46:05.325456 11116 tablet_service.cc:1511] Processing CreateTablet for tablet 6d83d6c1acf34440a37b9d2572a216da (DEFAULT_TABLE table=default.Table_9bcc86385a0840debdb02e97c0bd2144 [id=a96db20d48304ba6ad6413d78774c4b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.326506 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d83d6c1acf34440a37b9d2572a216da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.336692 11107 tablet_service.cc:1511] Processing CreateTablet for tablet 023d212fd21643caae5707984467c343 (DEFAULT_TABLE table=default.Table_597725134a0749bcb240facde21e0258 [id=39192535d7454f05b339b6a465b52917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.337661 11107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 023d212fd21643caae5707984467c343. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.328018 11551 consensus_queue.cc:260] T acc9c8e0f97e41b48a919aa579ad85cd P 33b5652538e64ed58a09993d3188f8d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.347427 10588 catalog_manager.cc:5811] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895) does not have the latest schema for tablet 63c1a610663a4a2fa58f6cdc5f8d86ce (table default.Table_bd6eedffdf6440efa3e776b7079015d8 [id=8a2084683a7740db80e2c52d3f4c9506]). Expected version 1 got 0
I20260809 01:46:05.357538 11127 tablet_service.cc:1511] Processing CreateTablet for tablet 59e28a41a4d34926b057d7b47882a4df (DEFAULT_TABLE table=default.Table_ff36f56ffe4640418291a7bc746caa81 [id=539bd89a047f4c0795ab1df710c021b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.298164 11336 tablet_service.cc:1511] Processing CreateTablet for tablet d887f9d14554452da644b0fb649af7b4 (DEFAULT_TABLE table=default.Table_2b0c8696f76f4997be51b001eaf80dc3 [id=58bde7931736495c82d90ea8d954d896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.298346 11554 tablet_bootstrap.cc:654] T 82d5a709cd86476594613e788a325094 P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:05.323419 11351 tablet_service.cc:1511] Processing CreateTablet for tablet 023d212fd21643caae5707984467c343 (DEFAULT_TABLE table=default.Table_597725134a0749bcb240facde21e0258 [id=39192535d7454f05b339b6a465b52917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.332870 11345 tablet_service.cc:1511] Processing CreateTablet for tablet 59e28a41a4d34926b057d7b47882a4df (DEFAULT_TABLE table=default.Table_ff36f56ffe4640418291a7bc746caa81 [id=539bd89a047f4c0795ab1df710c021b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.369932 11124 tablet_service.cc:1511] Processing CreateTablet for tablet d887f9d14554452da644b0fb649af7b4 (DEFAULT_TABLE table=default.Table_2b0c8696f76f4997be51b001eaf80dc3 [id=58bde7931736495c82d90ea8d954d896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.370338 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 023d212fd21643caae5707984467c343. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.374744 11345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59e28a41a4d34926b057d7b47882a4df. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:46:05.375300 11063 connection.cc:573] client connection to 127.10.8.65:41349 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:05.375732 11063 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 01:46:05.382679 11126 tablet_service.cc:1511] Processing CreateTablet for tablet c90494cd2d0d45e0bb4c6f8f941ccf62 (DEFAULT_TABLE table=default.Table_ba10600fdca746b3a5a091b2c78588a6 [id=fddc776bb73d470e96e4f8a206519062]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.383749 11123 tablet_service.cc:1511] Processing CreateTablet for tablet f5158431dbcf4eef91e9f4bc28d826fc (DEFAULT_TABLE table=default.Table_53624396796f4684acbc3787e6e05f6e [id=18e87d9e1fa3442aaa489e3abcd4640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.384087 13423 raft_consensus.cc:493] T a238ad3d46054daa9435e9076ba8311d P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:05.384577 13423 raft_consensus.cc:515] T a238ad3d46054daa9435e9076ba8311d P 33b5652538e64ed58a09993d3188f8d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.387627 13423 leader_election.cc:290] T a238ad3d46054daa9435e9076ba8311d P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
I20260809 01:46:05.389071 11392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a238ad3d46054daa9435e9076ba8311d" candidate_uuid: "33b5652538e64ed58a09993d3188f8d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f60c9ac9414403d9ea5a97b103c2316" is_pre_election: true
W20260809 01:46:05.390743 11062 leader_election.cc:343] T a238ad3d46054daa9435e9076ba8311d P 33b5652538e64ed58a09993d3188f8d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): Illegal state: must be running to vote when last-logged opid is not known
W20260809 01:46:05.392917 10580 connection.cc:573] client connection to 127.10.8.65:41349 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:05.393262 10580 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 4 similar messages]
I20260809 01:46:05.397769 11336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d887f9d14554452da644b0fb649af7b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.400400 11341 tablet_service.cc:1511] Processing CreateTablet for tablet c90494cd2d0d45e0bb4c6f8f941ccf62 (DEFAULT_TABLE table=default.Table_ba10600fdca746b3a5a091b2c78588a6 [id=fddc776bb73d470e96e4f8a206519062]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.401367 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c90494cd2d0d45e0bb4c6f8f941ccf62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.406594 11123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5158431dbcf4eef91e9f4bc28d826fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.412653 11126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c90494cd2d0d45e0bb4c6f8f941ccf62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.413623 13310 ts_tablet_manager.cc:1929] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:46:05.413997 13310 log.cc:1199] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/8323a7a40a1445f980c0adcf6a6d16c7
I20260809 01:46:05.414721 13310 ts_tablet_manager.cc:1950] T 8323a7a40a1445f980c0adcf6a6d16c7 P 33b5652538e64ed58a09993d3188f8d7: Deleting consensus metadata
I20260809 01:46:05.415977 11361 tablet_service.cc:1511] Processing CreateTablet for tablet f5158431dbcf4eef91e9f4bc28d826fc (DEFAULT_TABLE table=default.Table_53624396796f4684acbc3787e6e05f6e [id=18e87d9e1fa3442aaa489e3abcd4640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.416352 11551 ts_tablet_manager.cc:1434] T acc9c8e0f97e41b48a919aa579ad85cd P 33b5652538e64ed58a09993d3188f8d7: Time spent starting tablet: real 0.142s user 0.004s sys 0.001s
I20260809 01:46:05.416911 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5158431dbcf4eef91e9f4bc28d826fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.417277 11551 tablet_bootstrap.cc:492] T 19fe81e277cd4c798a07ceafa0b60566 P 33b5652538e64ed58a09993d3188f8d7: Bootstrap starting.
I20260809 01:46:05.419549 13310 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56566 (request call id 397) took 3601 ms. Trace:
I20260809 01:46:05.431190 11127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59e28a41a4d34926b057d7b47882a4df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.437944 11124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d887f9d14554452da644b0fb649af7b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.444402 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 76f0490d30e445c39cb1b072a5306f00 (DEFAULT_TABLE table=default.Table_4673a33b16564a8a8bd22ab3e30c3eec [id=dace5010b3cb4a63b9cb092750d7b481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.445416 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76f0490d30e445c39cb1b072a5306f00. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:46:05.450079 11289 connection.cc:573] client connection to 127.10.8.65:41349 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:05.450464 11289 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:05.454440 10579 connection.cc:573] server connection from 127.10.8.65:36985 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 01:46:05.454689 11551 tablet_bootstrap.cc:654] T 19fe81e277cd4c798a07ceafa0b60566 P 33b5652538e64ed58a09993d3188f8d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:05.419860 13310 rpcz_store.cc:276] 0809 01:46:01.825886 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:01.828393 (+ 2507us) service_pool.cc:224] Handling call
0809 01:46:05.090473 (+3262080us) ts_tablet_manager.cc:1151] Deleting tablet 8323a7a40a1445f980c0adcf6a6d16c7
0809 01:46:05.090541 (+ 68us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:05.277383 (+186842us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:05.369231 (+ 91848us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:05.419496 (+ 50265us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3261317}
I20260809 01:46:05.458596 10576 catalog_manager.cc:5050] TS 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): tablet 8323a7a40a1445f980c0adcf6a6d16c7 (table default.Table_78c91fd32f104ad785b008e345a24e04 [id=dc279b3b97e3489e803a3408d5d76395]) successfully deleted
I20260809 01:46:05.459630 13310 tablet_replica.cc:333] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7: stopping tablet replica
I20260809 01:46:05.460449 13310 raft_consensus.cc:2243] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:05.460879 13310 raft_consensus.cc:2272] T cd09bea5263344919ce26593fe958547 P 33b5652538e64ed58a09993d3188f8d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:05.463102 13161 ts_tablet_manager.cc:1929] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:46:05.463429 13161 log.cc:1199] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/8323a7a40a1445f980c0adcf6a6d16c7
I20260809 01:46:05.464104 13161 ts_tablet_manager.cc:1950] T 8323a7a40a1445f980c0adcf6a6d16c7 P 4f60c9ac9414403d9ea5a97b103c2316: Deleting consensus metadata
I20260809 01:46:05.471736 10273 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu with pid 10994
I20260809 01:46:05.482270 11347 tablet_service.cc:1511] Processing CreateTablet for tablet b5b1d75d377c4916b6646f7277d6bb97 (DEFAULT_TABLE table=default.Table_6133114bad9b4a68a734ecc6da250f6e [id=a743a0343e7a4d6c9f0a97350d5f9699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.483330 11347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5b1d75d377c4916b6646f7277d6bb97. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:46:05.493172 10580 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 99912.7us
W20260809 01:46:05.498903 11289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41349, user_credentials={real_user=slave}} blocked reactor thread for 48483.2us
I20260809 01:46:05.507419 11336 tablet_service.cc:1511] Processing CreateTablet for tablet 1b8475839a0641eab15b6fc2e0e38705 (DEFAULT_TABLE table=default.Table_4e15d84c0f2c4968878a65905971141b [id=5f3426f116a04d489c015b3048626ba8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.508448 11336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b8475839a0641eab15b6fc2e0e38705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.513972 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 413) took 3452 ms. Trace:
I20260809 01:46:05.514497 11554 tablet_bootstrap.cc:492] T 82d5a709cd86476594613e788a325094 P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:05.514426 13161 rpcz_store.cc:276] 0809 01:46:02.061102 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:02.061869 (+ 767us) service_pool.cc:224] Handling call
0809 01:46:05.199122 (+3137253us) ts_tablet_manager.cc:1151] Deleting tablet 8323a7a40a1445f980c0adcf6a6d16c7
0809 01:46:05.199159 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:05.366931 (+167772us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:05.462992 (+ 96061us) tablet_metadata.cc:665] Metadata flushed
0809 01:46:05.513920 (+ 50928us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3136686}
I20260809 01:46:05.515769 10579 catalog_manager.cc:5050] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): tablet 8323a7a40a1445f980c0adcf6a6d16c7 (table default.Table_78c91fd32f104ad785b008e345a24e04 [id=dc279b3b97e3489e803a3408d5d76395]) successfully deleted
I20260809 01:46:05.516815 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 416) took 3087 ms. Trace:
I20260809 01:46:05.514979 11554 ts_tablet_manager.cc:1403] T 82d5a709cd86476594613e788a325094 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.278s user 0.010s sys 0.009s
I20260809 01:46:05.519467 11554 raft_consensus.cc:359] T 82d5a709cd86476594613e788a325094 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.520072 11554 raft_consensus.cc:385] T 82d5a709cd86476594613e788a325094 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:05.520378 11554 raft_consensus.cc:740] T 82d5a709cd86476594613e788a325094 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:05.520979 11554 consensus_queue.cc:260] T 82d5a709cd86476594613e788a325094 P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
W20260809 01:46:05.526445 11289 consensus_peers.cc:597] T 59de95452c0a4945beae5a9344537e0f P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.65:41349: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 01:46:05.527457 11289 consensus_peers.cc:597] T fa12353c25cb4da680d478ce104bc6ba P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.65:41349: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 01:46:05.527809 11289 leader_election.cc:336] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Network error: Client connection negotiation failed: client connection to 127.10.8.65:41349: connect: Connection refused (error 111)
W20260809 01:46:05.533262 10580 catalog_manager.cc:4777] TS b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): CreateTablet RPC failed for tablet ee6dbf65ab424ba493f98cce6ca60672: Network error: Client connection negotiation failed: client connection to 127.10.8.65:41349: connect: Connection refused (error 111)
I20260809 01:46:05.530086 11289 leader_election.cc:304] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:05.537286 13466 raft_consensus.cc:2749] T e93334cc766044e79d37f293bc8556fb P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 01:46:05.538224 11289 consensus_peers.cc:597] T 7df6343423604a00a46f203f61e98e2f P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.65:41349: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 01:46:05.538693 11289 consensus_peers.cc:597] T cef88ffe9e95437da81793795742def3 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.65:41349: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 01:46:05.545686 11289 consensus_peers.cc:597] T 0354e4c7bf354f22a9c3c389e0e8ec92 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.65:41349: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 01:46:05.546129 11289 consensus_peers.cc:597] T cfb9afd3be194830ba99f5e20fdaca0a P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.65:41349: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 01:46:05.546510 11289 consensus_peers.cc:597] T 85d0a9d73d4841c393e7d2cef41fee89 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.65:41349: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 01:46:05.546921 11289 leader_election.cc:336] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Network error: Client connection negotiation failed: client connection to 127.10.8.65:41349: connect: Connection refused (error 111)
I20260809 01:46:05.547267 11289 leader_election.cc:304] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:05.547916 13466 raft_consensus.cc:3060] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:05.552266 13161 rpcz_store.cc:276] 0809 01:46:02.429745 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:02.434066 (+ 4321us) service_pool.cc:224] Handling call
0809 01:46:05.516506 (+3082440us) ts_tablet_manager.cc:1151] Deleting tablet 74d6b03590c8472db8977bee5962aa6c
0809 01:46:05.516546 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:05.516767 (+ 221us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3058330}
W20260809 01:46:05.553414 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet 74d6b03590c8472db8977bee5962aa6c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 74d6b03590c8472db8977bee5962aa6c already in progress: creating tablet
I20260809 01:46:05.553308 11341 tablet_service.cc:1511] Processing CreateTablet for tablet 195db925deed40a2b8007140e44747fd (DEFAULT_TABLE table=default.Table_76b0132a831f4433b712389f168fde06 [id=9af5cc13b5554064ada02900d4283e12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.554289 13466 raft_consensus.cc:2749] T afb6c244108a4ff3a2df0585e5280294 P 4f60c9ac9414403d9ea5a97b103c2316 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:05.554381 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 417) took 3118 ms. Trace:
I20260809 01:46:05.554381 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 195db925deed40a2b8007140e44747fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.554751 13161 rpcz_store.cc:276] 0809 01:46:02.435532 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:02.435636 (+ 104us) service_pool.cc:224] Handling call
0809 01:46:05.554104 (+3118468us) ts_tablet_manager.cc:1151] Deleting tablet c9e844ceeb364842b2aa3385c67963e7
0809 01:46:05.554136 (+ 32us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:05.554333 (+ 197us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3095744}
W20260809 01:46:05.555233 11289 consensus_peers.cc:597] T bb7a8b6d06064addb02979a7feedf756 P 4f60c9ac9414403d9ea5a97b103c2316 -> Peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Couldn't send request to peer b5117038a52a45b0aea3d30ca9cfcbe8. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.65:41349: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 01:46:05.555622 11289 leader_election.cc:336] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Network error: Client connection negotiation failed: client connection to 127.10.8.65:41349: connect: Connection refused (error 111)
W20260809 01:46:05.555886 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet c9e844ceeb364842b2aa3385c67963e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c9e844ceeb364842b2aa3385c67963e7 already in progress: creating tablet
W20260809 01:46:05.556241 11289 leader_election.cc:336] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Network error: Client connection negotiation failed: client connection to 127.10.8.65:41349: connect: Connection refused (error 111)
I20260809 01:46:05.556578 11289 leader_election.cc:304] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:05.557173 13466 raft_consensus.cc:2749] T c3bf43978dc94c2b9a7f3c18a4677c40 P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:05.557485 11554 ts_tablet_manager.cc:1434] T 82d5a709cd86476594613e788a325094 P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.040s user 0.003s sys 0.002s
W20260809 01:46:05.558210 11289 leader_election.cc:336] T 50f45c02a6994c5aaaf9c0a499e15fed P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Network error: Client connection negotiation failed: client connection to 127.10.8.65:41349: connect: Connection refused (error 111)
I20260809 01:46:05.582888 11361 tablet_service.cc:1511] Processing CreateTablet for tablet 6ca9ae4804ae4d98a8ea9b0bf1f43aeb (DEFAULT_TABLE table=default.Table_75b1acafcd514d78982b343ff92e9861 [id=9902923a811f43f78a96ba51db81d9fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.583837 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ca9ae4804ae4d98a8ea9b0bf1f43aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.610052 11554 tablet_bootstrap.cc:492] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
I20260809 01:46:05.563223 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 421) took 2791 ms. Trace:
I20260809 01:46:05.613793 13161 rpcz_store.cc:276] 0809 01:46:02.771708 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:02.771849 (+ 141us) service_pool.cc:224] Handling call
0809 01:46:05.562924 (+2791075us) ts_tablet_manager.cc:1151] Deleting tablet a238ad3d46054daa9435e9076ba8311d
0809 01:46:05.562964 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:05.563176 (+ 212us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2772999}
I20260809 01:46:05.614992 13161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53476 (request call id 427) took 2356 ms. Trace:
W20260809 01:46:05.615365 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet a238ad3d46054daa9435e9076ba8311d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a238ad3d46054daa9435e9076ba8311d already in progress: creating tablet
I20260809 01:46:05.615270 13161 rpcz_store.cc:276] 0809 01:46:03.258215 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:03.258390 (+ 175us) service_pool.cc:224] Handling call
0809 01:46:05.614683 (+2356293us) ts_tablet_manager.cc:1151] Deleting tablet b5bd08670e2e40e0bb250d5e054a4116
0809 01:46:05.614721 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 01:46:05.614945 (+ 224us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2350027}
I20260809 01:46:05.616200 13161 tablet_replica.cc:333] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316: stopping tablet replica
W20260809 01:46:05.616441 10579 catalog_manager.cc:5030] TS 4f60c9ac9414403d9ea5a97b103c2316 (127.10.8.67:40895): delete failed for tablet b5bd08670e2e40e0bb250d5e054a4116 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b5bd08670e2e40e0bb250d5e054a4116 already in progress: creating tablet
I20260809 01:46:05.616884 13161 raft_consensus.cc:2243] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:05.617295 13161 raft_consensus.cc:2272] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:05.664706 11341 tablet_service.cc:1511] Processing CreateTablet for tablet 212d15c0f6da43aab2ff26f815771bba (DEFAULT_TABLE table=default.Table_9c355bbef601486091e79dd70d877f1e [id=b30e345455fa4373aff0999861645a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.665259 11554 tablet_bootstrap.cc:654] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:05.665939 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 212d15c0f6da43aab2ff26f815771bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.679708 11347 tablet_service.cc:1511] Processing CreateTablet for tablet 5a0c8f777fd04632bad63bc25b55e1bb (DEFAULT_TABLE table=default.Table_b119819658d344a0887b87b4d540e8f2 [id=e9ce491fcee64ce5b176bd5f551be633]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.680696 11347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a0c8f777fd04632bad63bc25b55e1bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.669602 13161 ts_tablet_manager.cc:1916] T 9fffa48ccf58476da79dccd9c008b67c P 4f60c9ac9414403d9ea5a97b103c2316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:05.693639 11336 tablet_service.cc:1511] Processing CreateTablet for tablet 1732c5b25fb24c3d808c7d56c115f2b5 (DEFAULT_TABLE table=default.Table_921cdb8af3a2476cb5b87c0bf56902f9 [id=7e4ada019f4b47a08ff4ddcb865de12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.694554 11336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1732c5b25fb24c3d808c7d56c115f2b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.701804 11360 tablet_service.cc:1511] Processing CreateTablet for tablet d12ef7b08eae4978bcca5c3e8aa69a2d (DEFAULT_TABLE table=default.Table_6849525ffb55409998be560b07957dfe [id=b1ef18dd318a4815a34671efb702006d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.702757 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d12ef7b08eae4978bcca5c3e8aa69a2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.707427 11554 tablet_bootstrap.cc:492] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316: No bootstrap required, opened a new log
I20260809 01:46:05.707823 11554 ts_tablet_manager.cc:1403] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316: Time spent bootstrapping tablet: real 0.098s user 0.003s sys 0.005s
I20260809 01:46:05.735555 11361 tablet_service.cc:1511] Processing CreateTablet for tablet a1b924fd94b44d1e85480a2feeca57e6 (DEFAULT_TABLE table=default.Table_843b166662904713a045a2a0cf7f8bb2 [id=44abc4de32d4468da199ad42dc991cdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:05.736554 11361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1b924fd94b44d1e85480a2feeca57e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:05.738381 11554 raft_consensus.cc:359] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.738973 11554 raft_consensus.cc:385] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:05.739223 11554 raft_consensus.cc:740] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60c9ac9414403d9ea5a97b103c2316, State: Initialized, Role: FOLLOWER
I20260809 01:46:05.750556 11554 consensus_queue.cc:260] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.751201 13466 raft_consensus.cc:493] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:05.751549 13466 raft_consensus.cc:515] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33b5652538e64ed58a09993d3188f8d7" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 42495 } } peers { permanent_uuid: "4f60c9ac9414403d9ea5a97b103c2316" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 40895 } } peers { permanent_uuid: "b5117038a52a45b0aea3d30ca9cfcbe8" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41349 } }
I20260809 01:46:05.754700 13466 leader_election.cc:290] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495), b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349)
W20260809 01:46:05.759137 11289 leader_election.cc:336] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b5117038a52a45b0aea3d30ca9cfcbe8 (127.10.8.65:41349): Network error: Client connection negotiation failed: client connection to 127.10.8.65:41349: connect: Connection refused (error 111)
I20260809 01:46:05.774044 11554 ts_tablet_manager.cc:1434] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316: Time spent starting tablet: real 0.066s user 0.006s sys 0.001s
W20260809 01:46:05.798969 11289 connection.cc:573] server connection from 127.10.8.66:33591 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:05.799832 11283 connection.cc:573] client connection to 127.10.8.66:42495 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:05.800745 10576 connection.cc:573] client connection to 127.10.8.66:42495 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 01:46:05.801754 11554 tablet_bootstrap.cc:492] T f4d58189de244802b299e3b376319c1c P 4f60c9ac9414403d9ea5a97b103c2316: Bootstrap starting.
W20260809 01:46:05.806777 11283 leader_election.cc:336] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 33b5652538e64ed58a09993d3188f8d7 (127.10.8.66:42495): Network error: Client connection negotiation failed: client connection to 127.10.8.66:42495: connect: Connection refused (error 111)
I20260809 01:46:05.807160 11283 leader_election.cc:304] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f60c9ac9414403d9ea5a97b103c2316; no voters: 33b5652538e64ed58a09993d3188f8d7, b5117038a52a45b0aea3d30ca9cfcbe8
I20260809 01:46:05.807857 13466 raft_consensus.cc:2749] T 33b297f9083744e2953d41cd7b1055df P 4f60c9ac9414403d9ea5a97b103c2316 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:46:05.822695 10273 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu with pid 11251
W20260809 01:46:05.952957 10576 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.8.66:42495, user_credentials={real_user=slave}} blocked reactor thread for 59180.6us
W20260809 01:46:05.966677 10576 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 325e3d19b83545a197c2e09fc9643b5e (table default.Table_b21581ad07a34958a36ccbe1a72ab54d [id=ccd0ea07da604f859047f2eca85fc566], current schema version=4) failed: Not found: failed to reset TS proxy: No leader found: Tablet 325e3d19b83545a197c2e09fc9643b5e consensus state has no leader
I20260809 01:46:02.476872 12956 rpcz_store.cc:276] 0809 01:45:57.088669 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:45:57.089228 (+ 559us) service_pool.cc:224] Handling call
0809 01:45:58.071779 (+982551us) raft_consensus.cc:1471] Updating replica for 447 ops
0809 01:45:59.143089 (+1071310us) raft_consensus.cc:1534] Early marking committed up to index 6
0809 01:45:59.152105 (+ 9016us) raft_consensus.cc:1539] Triggering prepare for 447 ops
0809 01:45:59.152676 (+ 571us) write_op.cc:270] Start()
0809 01:45:59.152822 (+ 146us) write_op.cc:276] Timestamp: P: 1786239927431623 usec, L: 0
0809 01:45:59.165551 (+ 12729us) write_op.cc:270] Start()
0809 01:45:59.165821 (+ 270us) write_op.cc:276] Timestamp: P: 1786239927434249 usec, L: 0
0809 01:45:59.166649 (+ 828us) write_op.cc:270] Start()
0809 01:45:59.166761 (+ 112us) write_op.cc:276] Timestamp: P: 1786239927440092 usec, L: 0
0809 01:45:59.167546 (+ 785us) write_op.cc:270] Start()
0809 01:45:59.167651 (+ 105us) write_op.cc:276] Timestamp: P: 1786239927481607 usec, L: 0
0809 01:45:59.168656 (+ 1005us) write_op.cc:270] Start()
0809 01:45:59.168802 (+ 146us) write_op.cc:276] Timestamp: P: 1786239927496340 usec, L: 0
0809 01:45:59.169420 (+ 618us) write_op.cc:270] Start()
0809 01:45:59.169499 (+ 79us) write_op.cc:276] Timestamp: P: 1786239927529982 usec, L: 0
0809 01:45:59.170002 (+ 503us) write_op.cc:270] Start()
0809 01:45:59.170131 (+ 129us) write_op.cc:276] Timestamp: P: 1786239927549174 usec, L: 0
0809 01:45:59.170613 (+ 482us) write_op.cc:270] Start()
0809 01:45:59.170689 (+ 76us) write_op.cc:276] Timestamp: P: 1786239927563138 usec, L: 0
0809 01:45:59.171529 (+ 840us) write_op.cc:270] Start()
0809 01:45:59.171612 (+ 83us) write_op.cc:276] Timestamp: P: 1786239927599720 usec, L: 0
0809 01:45:59.191188 (+ 19576us) write_op.cc:270] Start()
0809 01:45:59.191350 (+ 162us) write_op.cc:276] Timestamp: P: 1786239927697099 usec, L: 0
0809 01:45:59.192076 (+ 726us) write_op.cc:270] Start()
0809 01:45:59.192222 (+ 146us) write_op.cc:276] Timestamp: P: 1786239927727036 usec, L: 0
0809 01:45:59.193056 (+ 834us) write_op.cc:270] Start()
0809 01:45:59.193254 (+ 198us) write_op.cc:276] Timestamp: P: 1786239927752856 usec, L: 0
0809 01:45:59.194267 (+ 1013us) write_op.cc:270] Start()
0809 01:45:59.195400 (+ 1133us) write_op.cc:276] Timestamp: P: 1786239927759067 usec, L: 0
0809 01:45:59.196106 (+ 706us) write_op.cc:270] Start()
0809 01:45:59.196246 (+ 140us) write_op.cc:276] Timestamp: P: 1786239927774554 usec, L: 0
0809 01:45:59.196974 (+ 728us) write_op.cc:270] Start()
0809 01:45:59.197068 (+ 94us) write_op.cc:276] Timestamp: P: 1786239927789708 usec, L: 0
0809 01:45:59.197673 (+ 605us) write_op.cc:270] Start()
0809 01:45:59.197816 (+ 143us) write_op.cc:276] Timestamp: P: 1786239927813474 usec, L: 0
0809 01:45:59.198681 (+ 865us) write_op.cc:270] Start()
0809 01:45:59.198803 (+ 122us) write_op.cc:276] Timestamp: P: 1786239927822367 usec, L: 0
0809 01:45:59.199588 (+ 785us) write_op.cc:270] Start()
0809 01:45:59.199730 (+ 142us) write_op.cc:276] Timestamp: P: 1786239927876862 usec, L: 0
0809 01:45:59.200506 (+ 776us) write_op.cc:270] Start()
0809 01:45:59.200611 (+ 105us) write_op.cc:276] Timestamp: P: 1786239927884100 usec, L: 0
0809 01:45:59.201157 (+ 546us) write_op.cc:270] Start()
0809 01:45:59.204330 (+ 3173us) write_op.cc:276] Timestamp: P: 1786239927979399 usec, L: 0
0809 01:45:59.205074 (+ 744us) write_op.cc:270] Start()
0809 01:45:59.205270 (+ 196us) write_op.cc:276] Timestamp: P: 1786239927987185 usec, L: 0
0809 01:45:59.205986 (+ 716us) write_op.cc:270] Start()
0809 01:45:59.206106 (+ 120us) write_op.cc:276] Timestamp: P: 1786239928050411 usec, L: 0
0809 01:45:59.206927 (+ 821us) write_op.cc:270] Start()
0809 01:45:59.207041 (+ 114us) write_op.cc:276] Timestamp: P: 1786239928080020 usec, L: 0
0809 01:45:59.207856 (+ 815us) write_op.cc:270] Start()
0809 01:45:59.208071 (+ 215us) write_op.cc:276] Timestamp: P: 1786239928087772 usec, L: 0
0809 01:45:59.209226 (+ 1155us) write_op.cc:270] Start()
0809 01:45:59.209371 (+ 145us) write_op.cc:276] Timestamp: P: 1786239928107585 usec, L: 0
0809 01:45:59.210005 (+ 634us) write_op.cc:270] Start()
0809 01:45:59.210122 (+ 117us) write_op.cc:276] Timestamp: P: 1786239928139959 usec, L: 0
0809 01:45:59.211167 (+ 1045us) write_op.cc:270] Start()
0809 01:45:59.211299 (+ 132us) write_op.cc:276] Timestamp: P: 1786239928156413 usec, L: 0
0809 01:45:59.212001 (+ 702us) write_op.cc:270] Start()
0809 01:45:59.212167 (+ 166us) write_op.cc:276] Timestamp: P: 1786239928199944 usec, L: 0
0809 01:45:59.212974 (+ 807us) write_op.cc:270] Start()
0809 01:45:59.213100 (+ 126us) write_op.cc:276] Timestamp: P: 1786239928206970 usec, L: 0
0809 01:45:59.213842 (+ 742us) write_op.cc:270] Start()
0809 01:45:59.214008 (+ 166us) write_op.cc:276] Timestamp: P: 1786239928279776 usec, L: 0
0809 01:45:59.214892 (+ 884us) write_op.cc:270] Start()
0809 01:45:59.214997 (+ 105us) write_op.cc:276] Timestamp: P: 1786239928318494 usec, L: 0
0809 01:45:59.215745 (+ 748us) write_op.cc:270] Start()
0809 01:45:59.215908 (+ 163us) write_op.cc:276] Timestamp: P: 1786239928323781 usec, L: 0
0809 01:45:59.216951 (+ 1043us) write_op.cc:270] Start()
0809 01:45:59.217135 (+ 184us) write_op.cc:276] Timestamp: P: 1786239928361263 usec, L: 0
0809 01:45:59.217784 (+ 649us) write_op.cc:270] Start()
0809 01:45:59.217899 (+ 115us) write_op.cc:276] Timestamp: P: 1786239928392234 usec, L: 0
0809 01:45:59.218695 (+ 796us) write_op.cc:270] Start()
0809 01:45:59.218845 (+ 150us) write_op.cc:276] Timestamp: P: 1786239928432921 usec, L: 0
0809 01:45:59.219640 (+ 795us) write_op.cc:270] Start()
0809 01:45:59.219796 (+ 156us) write_op.cc:276] Timestamp: P: 1786239928437529 usec, L: 0
0809 01:45:59.220586 (+ 790us) write_op.cc:270] Start()
0809 01:45:59.220725 (+ 139us) write_op.cc:276] Timestamp: P: 1786239928490419 usec, L: 0
0809 01:45:59.221794 (+ 1069us) write_op.cc:270] Start()
0809 01:45:59.221903 (+ 109us) write_op.cc:276] Timestamp: P: 1786239928579121 usec, L: 0
0809 01:45:59.222457 (+ 554us) write_op.cc:270] Start()
0809 01:45:59.222546 (+ 89us) write_op.cc:276] Timestamp: P: 1786239928598987 usec, L: 0
0809 01:45:59.223049 (+ 503us) write_op.cc:270] Start()
0809 01:45:59.223151 (+ 102us) write_op.cc:276] Timestamp: P: 1786239928618140 usec, L: 0
0809 01:45:59.223712 (+ 561us) write_op.cc:270] Start()
0809 01:45:59.223801 (+ 89us) write_op.cc:276] Timestamp: P: 1786239928669683 usec, L: 0
0809 01:45:59.224585 (+ 784us) write_op.cc:270] Start()
0809 01:45:59.224755 (+ 170us) write_op.cc:276] Timestamp: P: 1786239928675478 usec, L: 0
0809 01:45:59.225629 (+ 874us) write_op.cc:270] Start()
0809 01:45:59.225777 (+ 148us) write_op.cc:276] Timestamp: P: 1786239928827404 usec, L: 0
0809 01:45:59.226608 (+ 831us) write_op.cc:270] Start()
0809 01:45:59.226722 (+ 114us) write_op.cc:276] Timestamp: P: 1786239928854848 usec, L: 0
0809 01:45:59.227542 (+ 820us) write_op.cc:270] Start()
0809 01:45:59.227635 (+ 93us) write_op.cc:276] Timestamp: P: 1786239928873769 usec, L: 0
0809 01:45:59.228122 (+ 487us) write_op.cc:270] Start()
0809 01:45:59.228208 (+ 86us) write_op.cc:276] Timestamp: P: 1786239928996068 usec, L: 0
0809 01:45:59.228712 (+ 504us) write_op.cc:270] Start()
0809 01:45:59.228800 (+ 88us) write_op.cc:276] Timestamp: P: 1786239929007860 usec, L: 0
0809 01:45:59.229295 (+ 495us) write_op.cc:270] Start()
0809 01:45:59.229492 (+ 197us) write_op.cc:276] Timestamp: P: 1786239929014525 usec, L: 0
0809 01:45:59.230255 (+ 763us) write_op.cc:270] Start()
0809 01:45:59.230343 (+ 88us) write_op.cc:276] Timestamp: P: 1786239929023853 usec, L: 0
0809 01:45:59.230851 (+ 508us) write_op.cc:270] Start()
0809 01:45:59.230930 (+ 79us) write_op.cc:276] Timestamp: P: 1786239929039305 usec, L: 0
0809 01:45:59.231461 (+ 531us) write_op.cc:270] Start()
0809 01:45:59.231545 (+ 84us) write_op.cc:276] Timestamp: P: 1786239929288090 usec, L: 0
0809 01:45:59.232016 (+ 471us) write_op.cc:270] Start()
0809 01:45:59.232116 (+ 100us) write_op.cc:276] Timestamp: P: 1786239929674439 usec, L: 0
0809 01:45:59.232681 (+ 565us) write_op.cc:270] Start()
0809 01:45:59.232764 (+ 83us) write_op.cc:276] Timestamp: P: 1786239929677044 usec, L: 0
0809 01:45:59.233233 (+ 469us) write_op.cc:270] Start()
0809 01:45:59.233307 (+ 74us) write_op.cc:276] Timestamp: P: 1786239929772451 usec, L: 0
0809 01:45:59.233845 (+ 538us) write_op.cc:270] Start()
0809 01:45:59.233926 (+ 81us) write_op.cc:276] Timestamp: P: 1786239929855349 usec, L: 0
0809 01:45:59.234697 (+ 771us) write_op.cc:270] Start()
0809 01:45:59.234789 (+ 92us) write_op.cc:276] Timestamp: P: 1786239929859468 usec, L: 0
0809 01:45:59.235357 (+ 568us) write_op.cc:270] Start()
0809 01:45:59.235432 (+ 75us) write_op.cc:276] Timestamp: P: 1786239929862573 usec, L: 0
0809 01:45:59.235878 (+ 446us) write_op.cc:270] Start()
0809 01:45:59.235968 (+ 90us) write_op.cc:276] Timestamp: P: 1786239929865823 usec, L: 0
0809 01:45:59.236430 (+ 462us) write_op.cc:270] Start()
0809 01:45:59.236504 (+ 74us) write_op.cc:276] Timestamp: P: 1786239930046027 usec, L: 0
0809 01:45:59.236938 (+ 434us) write_op.cc:270] Start()
0809 01:45:59.237095 (+ 157us) write_op.cc:276] Timestamp: P: 1786239930130777 usec, L: 0
0809 01:45:59.237543 (+ 448us) write_op.cc:270] Start()
0809 01:45:59.237647 (+ 104us) write_op.cc:276] Timestamp: P: 1786239930139124 usec, L: 0
0809 01:45:59.238173 (+ 526us) write_op.cc:270] Start()
0809 01:45:59.238250 (+ 77us) write_op.cc:276] Timestamp: P: 1786239930147786 usec, L: 0
0809 01:45:59.238837 (+ 587us) write_op.cc:270] Start()
0809 01:45:59.238921 (+ 84us) write_op.cc:276] Timestamp: P: 1786239930244060 usec, L: 0
0809 01:45:59.239466 (+ 545us) write_op.cc:270] Start()
0809 01:45:59.239565 (+ 99us) write_op.cc:276] Timestamp: P: 1786239930305810 usec, L: 0
0809 01:45:59.241949 (+ 2384us) write_op.cc:270] Start()
0809 01:45:59.242024 (+ 75us) write_op.cc:276] Timestamp: P: 1786239930363399 usec, L: 0
0809 01:45:59.242525 (+ 501us) write_op.cc:270] Start()
0809 01:45:59.242598 (+ 73us) write_op.cc:276] Timestamp: P: 1786239930365928 usec, L: 0
0809 01:45:59.243036 (+ 438us) write_op.cc:270] Start()
0809 01:45:59.243107 (+ 71us) write_op.cc:276] Timestamp: P: 1786239930525944 usec, L: 0
0809 01:45:59.243516 (+ 409us) write_op.cc:270] Start()
0809 01:45:59.243585 (+ 69us) write_op.cc:276] Timestamp: P: 1786239930537248 usec, L: 0
0809 01:45:59.244054 (+ 469us) write_op.cc:270] Start()
0809 01:45:59.244129 (+ 75us) write_op.cc:276] Timestamp: P: 1786239930539698 usec, L: 0
0809 01:45:59.244549 (+ 420us) write_op.cc:270] Start()
0809 01:45:59.244621 (+ 72us) write_op.cc:276] Timestamp: P: 1786239930593239 usec, L: 0
0809 01:45:59.245084 (+ 463us) write_op.cc:270] Start()
0809 01:45:59.245154 (+ 70us) write_op.cc:276] Timestamp: P: 1786239930645816 usec, L: 0
0809 01:45:59.245641 (+ 487us) write_op.cc:270] Start()
0809 01:45:59.245719 (+ 78us) write_op.cc:276] Timestamp: P: 1786239930720053 usec, L: 0
0809 01:45:59.246214 (+ 495us) write_op.cc:270] Start()
0809 01:45:59.246289 (+ 75us) write_op.cc:276] Timestamp: P: 1786239930721823 usec, L: 0
0809 01:45:59.246729 (+ 440us) write_op.cc:270] Start()
0809 01:45:59.246800 (+ 71us) write_op.cc:276] Timestamp: P: 1786239930773276 usec, L: 0
0809 01:45:59.247234 (+ 434us) write_op.cc:270] Start()
0809 01:45:59.247309 (+ 75us) write_op.cc:276] Timestamp: P: 1786239930943636 usec, L: 0
0809 01:45:59.247731 (+ 422us) write_op.cc:270] Start()
0809 01:45:59.247828 (+ 97us) write_op.cc:276] Timestamp: P: 1786239930969474 usec, L: 0
0809 01:45:59.248303 (+ 475us) write_op.cc:270] Start()
0809 01:45:59.248384 (+ 81us) write_op.cc:276] Timestamp: P: 1786239930989307 usec, L: 0
0809 01:45:59.248841 (+ 457us) write_op.cc:270] Start()
0809 01:45:59.248929 (+ 88us) write_op.cc:276] Timestamp: P: 1786239931056457 usec, L: 0
0809 01:45:59.249410 (+ 481us) write_op.cc:270] Start()
0809 01:45:59.249487 (+ 77us) write_op.cc:276] Timestamp: P: 1786239931063457 usec, L: 0
0809 01:45:59.249967 (+ 480us) write_op.cc:270] Start()
0809 01:45:59.250044 (+ 77us) write_op.cc:276] Timestamp: P: 1786239931233324 usec, L: 0
0809 01:45:59.250468 (+ 424us) write_op.cc:270] Start()
0809 01:45:59.250548 (+ 80us) write_op.cc:276] Timestamp: P: 1786239931257222 usec, L: 0
0809 01:45:59.250982 (+ 434us) write_op.cc:270] Start()
0809 01:45:59.251063 (+ 81us) write_op.cc:276] Timestamp: P: 1786239931262046 usec, L: 0
0809 01:45:59.251527 (+ 464us) write_op.cc:270] Start()
0809 01:45:59.251599 (+ 72us) write_op.cc:276] Timestamp: P: 1786239931558870 usec, L: 0
0809 01:45:59.252066 (+ 467us) write_op.cc:270] Start()
0809 01:45:59.252148 (+ 82us) write_op.cc:276] Timestamp: P: 1786239931676460 usec, L: 0
0809 01:45:59.252600 (+ 452us) write_op.cc:270] Start()
0809 01:45:59.252680 (+ 80us) write_op.cc:276] Timestamp: P: 1786239931681073 usec, L: 0
0809 01:45:59.253151 (+ 471us) write_op.cc:270] Start()
0809 01:45:59.253231 (+ 80us) write_op.cc:276] Timestamp: P: 1786239931791032 usec, L: 0
0809 01:45:59.253785 (+ 554us) write_op.cc:270] Start()
0809 01:45:59.253876 (+ 91us) write_op.cc:276] Timestamp: P: 1786239931844501 usec, L: 0
0809 01:45:59.254345 (+ 469us) write_op.cc:270] Start()
0809 01:45:59.254419 (+ 74us) write_op.cc:276] Timestamp: P: 1786239931911446 usec, L: 0
0809 01:45:59.254872 (+ 453us) write_op.cc:270] Start()
0809 01:45:59.254951 (+ 79us) write_op.cc:276] Timestamp: P: 1786239931947463 usec, L: 0
0809 01:45:59.255337 (+ 386us) write_op.cc:270] Start()
0809 01:45:59.255401 (+ 64us) write_op.cc:276] Timestamp: P: 1786239931994907 usec, L: 0
0809 01:45:59.255789 (+ 388us) write_op.cc:270] Start()
0809 01:45:59.255855 (+ 66us) write_op.cc:276] Timestamp: P: 1786239932007426 usec, L: 0
0809 01:45:59.256279 (+ 424us) write_op.cc:270] Start()
0809 01:45:59.256346 (+ 67us) write_op.cc:276] Timestamp: P: 1786239932044449 usec, L: 0
0809 01:45:59.256820 (+ 474us) write_op.cc:270] Start()
0809 01:45:59.256904 (+ 84us) write_op.cc:276] Timestamp: P: 1786239932078396 usec, L: 0
0809 01:45:59.257325 (+ 421us) write_op.cc:270] Start()
0809 01:45:59.257393 (+ 68us) write_op.cc:276] Timestamp: P: 1786239932285607 usec, L: 0
0809 01:45:59.257853 (+ 460us) write_op.cc:270] Start()
0809 01:45:59.257928 (+ 75us) write_op.cc:276] Timestamp: P: 1786239932351368 usec, L: 0
0809 01:45:59.258397 (+ 469us) write_op.cc:270] Start()
0809 01:45:59.258466 (+ 69us) write_op.cc:276] Timestamp: P: 1786239932376963 usec, L: 0
0809 01:45:59.258874 (+ 408us) write_op.cc:270] Start()
0809 01:45:59.258955 (+ 81us) write_op.cc:276] Timestamp: P: 1786239932416202 usec, L: 0
0809 01:45:59.259355 (+ 400us) write_op.cc:270] Start()
0809 01:45:59.259506 (+ 151us) write_op.cc:276] Timestamp: P: 1786239932431652 usec, L: 0
0809 01:45:59.260036 (+ 530us) write_op.cc:270] Start()
0809 01:45:59.260108 (+ 72us) write_op.cc:276] Timestamp: P: 1786239932457749 usec, L: 0
0809 01:45:59.260495 (+ 387us) write_op.cc:270] Start()
0809 01:45:59.260560 (+ 65us) write_op.cc:276] Timestamp: P: 1786239932464932 usec, L: 0
0809 01:45:59.270969 (+ 10409us) write_op.cc:270] Start()
0809 01:45:59.271041 (+ 72us) write_op.cc:276] Timestamp: P: 1786239932545709 usec, L: 0
0809 01:45:59.271457 (+ 416us) write_op.cc:270] Start()
0809 01:45:59.271522 (+ 65us) write_op.cc:276] Timestamp: P: 1786239932820798 usec, L: 0
0809 01:45:59.271947 (+ 425us) write_op.cc:270] Start()
0809 01:45:59.272103 (+ 156us) write_op.cc:276] Timestamp: P: 1786239932848357 usec, L: 0
0809 01:45:59.272566 (+ 463us) write_op.cc:270] Start()
0809 01:45:59.272644 (+ 78us) write_op.cc:276] Timestamp: P: 1786239932852388 usec, L: 0
0809 01:45:59.273080 (+ 436us) write_op.cc:270] Start()
0809 01:45:59.273149 (+ 69us) write_op.cc:276] Timestamp: P: 1786239932904681 usec, L: 0
0809 01:45:59.273543 (+ 394us) write_op.cc:270] Start()
0809 01:45:59.273606 (+ 63us) write_op.cc:276] Timestamp: P: 1786239932979856 usec, L: 0
0809 01:45:59.274016 (+ 410us) write_op.cc:270] Start()
0809 01:45:59.274076 (+ 60us) write_op.cc:276] Timestamp: P: 1786239933215571 usec, L: 0
0809 01:45:59.274439 (+ 363us) write_op.cc:270] Start()
0809 01:45:59.274502 (+ 63us) write_op.cc:276] Timestamp: P: 1786239933242869 usec, L: 0
0809 01:45:59.274914 (+ 412us) write_op.cc:270] Start()
0809 01:45:59.274976 (+ 62us) write_op.cc:276] Timestamp: P: 1786239933247433 usec, L: 0
0809 01:45:59.275336 (+ 360us) write_op.cc:270] Start()
0809 01:45:59.275395 (+ 59us) write_op.cc:276] Timestamp: P: 1786239933407463 usec, L: 0
0809 01:45:59.275755 (+ 360us) write_op.cc:270] Start()
0809 01:45:59.275815 (+ 60us) write_op.cc:276] Timestamp: P: 1786239933421756 usec, L: 0
0809 01:45:59.276177 (+ 362us) write_op.cc:270] Start()
0809 01:45:59.276241 (+ 64us) write_op.cc:276] Timestamp: P: 1786239933570039 usec, L: 0
0809 01:45:59.276636 (+ 395us) write_op.cc:270] Start()
0809 01:45:59.276703 (+ 67us) write_op.cc:276] Timestamp: P: 1786239933591373 usec, L: 0
0809 01:45:59.277088 (+ 385us) write_op.cc:270] Start()
0809 01:45:59.277161 (+ 73us) write_op.cc:276] Timestamp: P: 1786239933628720 usec, L: 0
0809 01:45:59.277568 (+ 407us) write_op.cc:270] Start()
0809 01:45:59.277643 (+ 75us) write_op.cc:276] Timestamp: P: 1786239933686886 usec, L: 0
0809 01:45:59.278093 (+ 450us) write_op.cc:270] Start()
0809 01:45:59.278160 (+ 67us) write_op.cc:276] Timestamp: P: 1786239933773286 usec, L: 0
0809 01:45:59.278575 (+ 415us) write_op.cc:270] Start()
0809 01:45:59.278639 (+ 64us) write_op.cc:276] Timestamp: P: 1786239933777818 usec, L: 0
0809 01:45:59.279043 (+ 404us) write_op.cc:270] Start()
0809 01:45:59.279120 (+ 77us) write_op.cc:276] Timestamp: P: 1786239933780662 usec, L: 0
0809 01:45:59.279509 (+ 389us) write_op.cc:270] Start()
0809 01:45:59.279570 (+ 61us) write_op.cc:276] Timestamp: P: 1786239933802580 usec, L: 0
0809 01:45:59.279955 (+ 385us) write_op.cc:270] Start()
0809 01:45:59.280019 (+ 64us) write_op.cc:276] Timestamp: P: 1786239933842918 usec, L: 0
0809 01:45:59.280447 (+ 428us) write_op.cc:270] Start()
0809 01:45:59.280510 (+ 63us) write_op.cc:276] Timestamp: P: 1786239934005583 usec, L: 0
0809 01:45:59.280873 (+ 363us) write_op.cc:270] Start()
0809 01:45:59.280944 (+ 71us) write_op.cc:276] Timestamp: P: 1786239934296772 usec, L: 0
0809 01:45:59.281331 (+ 387us) write_op.cc:270] Start()
0809 01:45:59.281402 (+ 71us) write_op.cc:276] Timestamp: P: 1786239934313150 usec, L: 0
0809 01:45:59.281787 (+ 385us) write_op.cc:270] Start()
0809 01:45:59.281848 (+ 61us) write_op.cc:276] Timestamp: P: 1786239934320965 usec, L: 0
0809 01:45:59.282290 (+ 442us) write_op.cc:270] Start()
0809 01:45:59.282369 (+ 79us) write_op.cc:276] Timestamp: P: 1786239934324125 usec, L: 0
0809 01:45:59.282765 (+ 396us) write_op.cc:270] Start()
0809 01:45:59.282828 (+ 63us) write_op.cc:276] Timestamp: P: 1786239934632787 usec, L: 0
0809 01:45:59.283435 (+ 607us) write_op.cc:270] Start()
0809 01:45:59.283511 (+ 76us) write_op.cc:276] Timestamp: P: 1786239934875867 usec, L: 0
0809 01:45:59.284013 (+ 502us) write_op.cc:270] Start()
0809 01:45:59.284092 (+ 79us) write_op.cc:276] Timestamp: P: 1786239934881663 usec, L: 0
0809 01:45:59.284726 (+ 634us) write_op.cc:270] Start()
0809 01:45:59.284799 (+ 73us) write_op.cc:276] Timestamp: P: 1786239934883596 usec, L: 0
0809 01:45:59.285272 (+ 473us) write_op.cc:270] Start()
0809 01:45:59.285346 (+ 74us) write_op.cc:276] Timestamp: P: 1786239934920459 usec, L: 0
0809 01:45:59.285798 (+ 452us) write_op.cc:270] Start()
0809 01:45:59.285952 (+ 154us) write_op.cc:276] Timestamp: P: 1786239934974178 usec, L: 0
0809 01:45:59.286379 (+ 427us) write_op.cc:270] Start()
0809 01:45:59.286460 (+ 81us) write_op.cc:276] Timestamp: P: 1786239934976667 usec, L: 0
0809 01:45:59.286918 (+ 458us) write_op.cc:270] Start()
0809 01:45:59.287001 (+ 83us) write_op.cc:276] Timestamp: P: 1786239934979235 usec, L: 0
0809 01:45:59.287409 (+ 408us) write_op.cc:270] Start()
0809 01:45:59.287478 (+ 69us) write_op.cc:276] Timestamp: P: 1786239934982269 usec, L: 0
0809 01:45:59.287924 (+ 446us) write_op.cc:270] Start()
0809 01:45:59.287994 (+ 70us) write_op.cc:276] Timestamp: P: 1786239935067075 usec, L: 0
0809 01:45:59.288432 (+ 438us) write_op.cc:270] Start()
0809 01:45:59.288513 (+ 81us) write_op.cc:276] Timestamp: P: 1786239935247741 usec, L: 0
0809 01:45:59.288947 (+ 434us) write_op.cc:270] Start()
0809 01:45:59.289017 (+ 70us) write_op.cc:276] Timestamp: P: 1786239935324344 usec, L: 0
0809 01:45:59.289453 (+ 436us) write_op.cc:270] Start()
0809 01:45:59.289517 (+ 64us) write_op.cc:276] Timestamp: P: 1786239935413612 usec, L: 0
0809 01:45:59.289947 (+ 430us) write_op.cc:270] Start()
0809 01:45:59.290018 (+ 71us) write_op.cc:276] Timestamp: P: 1786239935498520 usec, L: 0
0809 01:45:59.290412 (+ 394us) write_op.cc:270] Start()
0809 01:45:59.290489 (+ 77us) write_op.cc:276] Timestamp: P: 1786239935525901 usec, L: 0
0809 01:45:59.290903 (+ 414us) write_op.cc:270] Start()
0809 01:45:59.290967 (+ 64us) write_op.cc:276] Timestamp: P: 1786239935624534 usec, L: 0
0809 01:45:59.291390 (+ 423us) write_op.cc:270] Start()
0809 01:45:59.291457 (+ 67us) write_op.cc:276] Timestamp: P: 1786239935673719 usec, L: 0
0809 01:45:59.291853 (+ 396us) write_op.cc:270] Start()
0809 01:45:59.291927 (+ 74us) write_op.cc:276] Timestamp: P: 1786239935790733 usec, L: 0
0809 01:45:59.292297 (+ 370us) write_op.cc:270] Start()
0809 01:45:59.292358 (+ 61us) write_op.cc:276] Timestamp: P: 1786239935928231 usec, L: 0
0809 01:45:59.292742 (+ 384us) write_op.cc:270] Start()
0809 01:45:59.292806 (+ 64us) write_op.cc:276] Timestamp: P: 1786239935988215 usec, L: 0
0809 01:45:59.293187 (+ 381us) write_op.cc:270] Start()
0809 01:45:59.293257 (+ 70us) write_op.cc:276] Timestamp: P: 1786239936016368 usec, L: 0
0809 01:45:59.293661 (+ 404us) write_op.cc:270] Start()
0809 01:45:59.293727 (+ 66us) write_op.cc:276] Timestamp: P: 1786239936045237 usec, L: 0
0809 01:45:59.294125 (+ 398us) write_op.cc:270] Start()
0809 01:45:59.294188 (+ 63us) write_op.cc:276] Timestamp: P: 1786239936112364 usec, L: 0
0809 01:45:59.294553 (+ 365us) write_op.cc:270] Start()
0809 01:45:59.294614 (+ 61us) write_op.cc:276] Timestamp: P: 1786239936123601 usec, L: 0
0809 01:45:59.294982 (+ 368us) write_op.cc:270] Start()
0809 01:45:59.295046 (+ 64us) write_op.cc:276] Timestamp: P: 1786239936185923 usec, L: 0
0809 01:45:59.295487 (+ 441us) write_op.cc:270] Start()
0809 01:45:59.295555 (+ 68us) write_op.cc:276] Timestamp: P: 1786239936397712 usec, L: 0
0809 01:45:59.295983 (+ 428us) write_op.cc:270] Start()
0809 01:45:59.296046 (+ 63us) write_op.cc:276] Timestamp: P: 1786239936406357 usec, L: 0
0809 01:45:59.296506 (+ 460us) write_op.cc:270] Start()
0809 01:45:59.296582 (+ 76us) write_op.cc:276] Timestamp: P: 1786239936414282 usec, L: 0
0809 01:45:59.297012 (+ 430us) write_op.cc:270] Start()
0809 01:45:59.297083 (+ 71us) write_op.cc:276] Timestamp: P: 1786239936491943 usec, L: 0
0809 01:45:59.297496 (+ 413us) write_op.cc:270] Start()
0809 01:45:59.297585 (+ 89us) write_op.cc:276] Timestamp: P: 1786239936626835 usec, L: 0
0809 01:45:59.297994 (+ 409us) write_op.cc:270] Start()
0809 01:45:59.298064 (+ 70us) write_op.cc:276] Timestamp: P: 1786239936655244 usec, L: 0
0809 01:45:59.298503 (+ 439us) write_op.cc:270] Start()
0809 01:45:59.298583 (+ 80us) write_op.cc:276] Timestamp: P: 1786239936730500 usec, L: 0
0809 01:45:59.299056 (+ 473us) write_op.cc:270] Start()
0809 01:45:59.299125 (+ 69us) write_op.cc:276] Timestamp: P: 1786239936733263 usec, L: 0
0809 01:45:59.299535 (+ 410us) write_op.cc:270] Start()
0809 01:45:59.299599 (+ 64us) write_op.cc:276] Timestamp: P: 1786239937268369 usec, L: 0
0809 01:45:59.299993 (+ 394us) write_op.cc:270] Start()
0809 01:45:59.300068 (+ 75us) write_op.cc:276] Timestamp: P: 1786239937299948 usec, L: 0
0809 01:45:59.300456 (+ 388us) write_op.cc:270] Start()
0809 01:45:59.300528 (+ 72us) write_op.cc:276] Timestamp: P: 1786239937430583 usec, L: 0
0809 01:45:59.300959 (+ 431us) write_op.cc:270] Start()
0809 01:45:59.301027 (+ 68us) write_op.cc:276] Timestamp: P: 1786239937444870 usec, L: 0
0809 01:45:59.301442 (+ 415us) write_op.cc:270] Start()
0809 01:45:59.301509 (+ 67us) write_op.cc:276] Timestamp: P: 1786239937462552 usec, L: 0
0809 01:45:59.301947 (+ 438us) write_op.cc:270] Start()
0809 01:45:59.302017 (+ 70us) write_op.cc:276] Timestamp: P: 1786239937472777 usec, L: 0
0809 01:45:59.302951 (+ 934us) write_op.cc:270] Start()
0809 01:45:59.303027 (+ 76us) write_op.cc:276] Timestamp: P: 1786239937679442 usec, L: 0
0809 01:45:59.303473 (+ 446us) write_op.cc:270] Start()
0809 01:45:59.303546 (+ 73us) write_op.cc:276] Timestamp: P: 1786239937682912 usec, L: 0
0809 01:45:59.304049 (+ 503us) write_op.cc:270] Start()
0809 01:45:59.304143 (+ 94us) write_op.cc:276] Timestamp: P: 1786239937731044 usec, L: 0
0809 01:45:59.304510 (+ 367us) write_op.cc:270] Start()
0809 01:45:59.304577 (+ 67us) write_op.cc:276] Timestamp: P: 1786239937796297 usec, L: 0
0809 01:45:59.305008 (+ 431us) write_op.cc:270] Start()
0809 01:45:59.305079 (+ 71us) write_op.cc:276] Timestamp: P: 1786239937812136 usec, L: 0
0809 01:45:59.400149 (+ 95070us) write_op.cc:270] Start()
0809 01:45:59.400233 (+ 84us) write_op.cc:276] Timestamp: P: 1786239937821398 usec, L: 0
0809 01:45:59.400754 (+ 521us) write_op.cc:270] Start()
0809 01:45:59.400835 (+ 81us) write_op.cc:276] Timestamp: P: 1786239937826360 usec, L: 0
0809 01:45:59.401278 (+ 443us) write_op.cc:270] Start()
0809 01:45:59.401345 (+ 67us) write_op.cc:276] Timestamp: P: 1786239937883745 usec, L: 0
0809 01:45:59.413124 (+ 11779us) write_op.cc:270] Start()
0809 01:45:59.413213 (+ 89us) write_op.cc:276] Timestamp: P: 1786239937904559 usec, L: 0
0809 01:45:59.416277 (+ 3064us) write_op.cc:270] Start()
0809 01:45:59.416347 (+ 70us) write_op.cc:276] Timestamp: P: 1786239938062924 usec, L: 0
0809 01:45:59.416778 (+ 431us) write_op.cc:270] Start()
0809 01:45:59.416848 (+ 70us) write_op.cc:276] Timestamp: P: 1786239938080363 usec, L: 0
0809 01:45:59.417304 (+ 456us) write_op.cc:270] Start()
0809 01:45:59.417389 (+ 85us) write_op.cc:276] Timestamp: P: 1786239938216936 usec, L: 0
0809 01:45:59.417902 (+ 513us) write_op.cc:270] Start()
0809 01:45:59.417978 (+ 76us) write_op.cc:276] Timestamp: P: 1786239938293714 usec, L: 0
0809 01:45:59.418515 (+ 537us) write_op.cc:270] Start()
0809 01:45:59.418591 (+ 76us) write_op.cc:276] Timestamp: P: 1786239938295608 usec, L: 0
0809 01:45:59.419092 (+ 501us) write_op.cc:270] Start()
0809 01:45:59.419177 (+ 85us) write_op.cc:276] Timestamp: P: 1786239938381454 usec, L: 0
0809 01:45:59.419797 (+ 620us) write_op.cc:270] Start()
0809 01:45:59.419875 (+ 78us) write_op.cc:276] Timestamp: P: 1786239938385037 usec, L: 0
0809 01:45:59.420345 (+ 470us) write_op.cc:270] Start()
0809 01:45:59.420426 (+ 81us) write_op.cc:276] Timestamp: P: 1786239938387033 usec, L: 0
0809 01:45:59.423034 (+ 2608us) write_op.cc:270] Start()
0809 01:45:59.423120 (+ 86us) write_op.cc:276] Timestamp: P: 1786239938459782 usec, L: 0
0809 01:45:59.423606 (+ 486us) write_op.cc:270] Start()
0809 01:45:59.423704 (+ 98us) write_op.cc:276] Timestamp: P: 1786239938555519 usec, L: 0
0809 01:45:59.424153 (+ 449us) write_op.cc:270] Start()
0809 01:45:59.424226 (+ 73us) write_op.cc:276] Timestamp: P: 1786239938583729 usec, L: 0
0809 01:45:59.424670 (+ 444us) write_op.cc:270] Start()
0809 01:45:59.424745 (+ 75us) write_op.cc:276] Timestamp: P: 1786239938669146 usec, L: 0
0809 01:45:59.425262 (+ 517us) write_op.cc:270] Start()
0809 01:45:59.425339 (+ 77us) write_op.cc:276] Timestamp: P: 1786239938725475 usec, L: 0
0809 01:45:59.425805 (+ 466us) write_op.cc:270] Start()
0809 01:45:59.425882 (+ 77us) write_op.cc:276] Timestamp: P: 1786239938728694 usec, L: 0
0809 01:45:59.426295 (+ 413us) write_op.cc:270] Start()
0809 01:45:59.426373 (+ 78us) write_op.cc:276] Timestamp: P: 1786239938779719 usec, L: 0
0809 01:45:59.426772 (+ 399us) write_op.cc:270] Start()
0809 01:45:59.426851 (+ 79us) write_op.cc:276] Timestamp: P: 1786239938787340 usec, L: 0
0809 01:45:59.427284 (+ 433us) write_op.cc:270] Start()
0809 01:45:59.427351 (+ 67us) write_op.cc:276] Timestamp: P: 1786239938812893 usec, L: 0
0809 01:45:59.461507 (+ 34156us) write_op.cc:270] Start()
0809 01:45:59.461618 (+ 111us) write_op.cc:276] Timestamp: P: 1786239938833741 usec, L: 0
0809 01:45:59.462277 (+ 659us) write_op.cc:270] Start()
0809 01:45:59.462364 (+ 87us) write_op.cc:276] Timestamp: P: 1786239938858225 usec, L: 0
0809 01:45:59.468098 (+ 5734us) write_op.cc:270] Start()
0809 01:45:59.468180 (+ 82us) write_op.cc:276] Timestamp: P: 1786239938863953 usec, L: 0
0809 01:45:59.468648 (+ 468us) write_op.cc:270] Start()
0809 01:45:59.468723 (+ 75us) write_op.cc:276] Timestamp: P: 1786239938926666 usec, L: 0
0809 01:45:59.469211 (+ 488us) write_op.cc:270] Start()
0809 01:45:59.469292 (+ 81us) write_op.cc:276] Timestamp: P: 1786239938930271 usec, L: 0
0809 01:45:59.469725 (+ 433us) write_op.cc:270] Start()
0809 01:45:59.479231 (+ 9506us) write_op.cc:276] Timestamp: P: 1786239938990996 usec, L: 0
0809 01:45:59.479822 (+ 591us) write_op.cc:270] Start()
0809 01:45:59.479917 (+ 95us) write_op.cc:276] Timestamp: P: 1786239
W20260809 01:46:06.090005 10579 connection.cc:573] client connection to 127.10.8.67:40895 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:06.089895 12870 connection.cc:573] server connection from 127.10.8.67:36721 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:06.090821 10319 connection.cc:573] server connection from 127.10.8.67:34281 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 01:46:06.100535 10273 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu with pid 10289
I20260809 01:46:06.108229 13499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "629f002048f64491afdd57f72a466e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a73ee8e8ec496a984efcab27715506" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 42157 } } peers { permanent_uuid: "71bc72deb1d2402cabd817dfe3b27e88" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 42063 } } peers { permanent_uuid: "629f002048f64491afdd57f72a466e59" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 33929 } } }
I20260809 01:46:06.117858 13499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 [sys.catalog]: This master's current role is: LEADER
I20260809 01:46:06.180280 10273 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu with pid 11743
W20260809 01:46:06.202718 10576 consensus_peers.cc:597] T 00000000000000000000000000000000 P 629f002048f64491afdd57f72a466e59 -> Peer 09a73ee8e8ec496a984efcab27715506 (127.10.8.126:42157): Couldn't send request to peer 09a73ee8e8ec496a984efcab27715506. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.126:42157: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 01:46:06.287662 10581 connection.cc:573] client connection to 127.10.8.125:42063 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 01:46:06.287992 10273 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu with pid 10520
2026-08-09T01:46:06Z chronyd exiting
I20260809 01:46:06.362673 10273 master-stress-test.cc:587] Tables created: 258
I20260809 01:46:06.362893 10273 master-stress-test.cc:588] Tables altered: 124
I20260809 01:46:06.363080 10273 master-stress-test.cc:589] Tables deleted: 86
I20260809 01:46:06.363247 10273 master-stress-test.cc:590] Tablets replaced: 44
I20260809 01:46:06.363410 10273 master-stress-test.cc:591] Masters restarted: 1
[ OK ] CatalogManagerConfigurations/MasterStressTest.Test/0 (57165 ms)
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/3
2026-08-09T01:46:06Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:46:06Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskctLAn3/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskctLAn3/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/security.properties
2026-08-09 01:46:13: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskctLAn3/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskctLAn3/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260809 01:46:22.721828 10273 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
/tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.126:43137
--webserver_interface=127.10.8.126
--webserver_port=0
--builtin_ntp_servers=127.10.8.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--hive_metastore_uris=thrift://127.0.0.1:35729
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260809 01:46:23.106474 13942 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:46:23.107090 13942 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:46:23.107568 13942 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 01:46:23.140975 13942 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260809 01:46:23.141271 13942 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260809 01:46:23.141572 13942 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:46:23.141844 13942 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:46:23.142112 13942 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:46:23.142407 13942 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:46:23.179207 13942 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.8.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:35729
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.126:43137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.8.126
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 01:43:22 UTC on e7f111948823
build id 13251
TSAN enabled
I20260809 01:46:23.180609 13942 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:46:23.182402 13942 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 01:46:23.192689 13953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:23.197772 13959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:23.194157 13954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:24.371939 13956 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260809 01:46:24.372077 13942 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260809 01:46:24.375684 13942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:46:24.378975 13942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:46:24.380426 13942 hybrid_clock.cc:648] HybridClock initialized: now 1786239984380386 us; error 55 us; skew 500 ppm
I20260809 01:46:24.381346 13942 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:46:24.388360 13942 webserver.cc:533] Webserver started at http://127.10.8.126:45899/ using document root <none> and password file <none>
I20260809 01:46:24.389323 13942 fs_manager.cc:362] Metadata directory not provided
I20260809 01:46:24.389587 13942 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:46:24.390092 13942 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:46:24.394542 13942 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/data/instance:
uuid: "cdec945a08924d01a5c58babffef9e31"
format_stamp: "Formatted at 2026-08-09 01:46:24 on dist-test-slave-zr1t"
I20260809 01:46:24.395680 13942 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/wal/instance:
uuid: "cdec945a08924d01a5c58babffef9e31"
format_stamp: "Formatted at 2026-08-09 01:46:24 on dist-test-slave-zr1t"
I20260809 01:46:24.403507 13942 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.001s sys 0.008s
I20260809 01:46:24.409469 13971 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:46:24.410657 13942 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 01:46:24.411018 13942 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/data,/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/wal
uuid: "cdec945a08924d01a5c58babffef9e31"
format_stamp: "Formatted at 2026-08-09 01:46:24 on dist-test-slave-zr1t"
I20260809 01:46:24.411381 13942 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:46:24.463892 13942 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 01:46:24.465441 13942 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:46:24.465957 13942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:46:24.541747 13942 rpc_server.cc:307] RPC server started. Bound to: 127.10.8.126:43137
I20260809 01:46:24.541868 14045 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.8.126:43137 every 8 connection(s)
I20260809 01:46:24.544592 13942 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/data/info.pb
I20260809 01:46:24.550642 10273 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu as pid 13942
I20260809 01:46:24.551054 14047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:24.551194 10273 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-0/wal/instance
I20260809 01:46:24.555079 10273 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
/tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.125:43535
--webserver_interface=127.10.8.125
--webserver_port=0
--builtin_ntp_servers=127.10.8.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--hive_metastore_uris=thrift://127.0.0.1:35729
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260809 01:46:24.568298 14047 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:46:24.593410 14047 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } has no permanent_uuid. Determining permanent_uuid...
W20260809 01:46:24.596056 13975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:46:24.600772 14047 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.125:43535: Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:46:24.650305 14047 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } attempt: 1
W20260809 01:46:24.654654 14047 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.125:43535: Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:46:24.723063 14047 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } attempt: 2
W20260809 01:46:24.727818 14047 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.125:43535: Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:46:24.819293 14047 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } attempt: 3
W20260809 01:46:24.823781 14047 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.125:43535: Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:46:24.883733 14049 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:46:24.884323 14049 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:46:24.884838 14049 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 01:46:24.918053 14049 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260809 01:46:24.918304 14049 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260809 01:46:24.918507 14049 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:46:24.918706 14049 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:46:24.918913 14049 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:46:24.919113 14049 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:46:24.957165 14049 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.8.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:35729
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.125:43535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.8.125
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 01:43:22 UTC on e7f111948823
build id 13251
TSAN enabled
I20260809 01:46:24.958551 14049 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:46:24.960213 14049 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 01:46:24.967223 14047 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } attempt: 4
W20260809 01:46:24.972357 14062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:24.975206 14047 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.125:43535: Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:46:25.274684 14047 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } attempt: 5
W20260809 01:46:25.282128 14047 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.125:43535: Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:46:25.829825 14047 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } attempt: 6
W20260809 01:46:25.841481 14047 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.125:43535: Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:46:24.974071 14064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:26.205180 14068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:26.207628 14065 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1230 milliseconds
I20260809 01:46:26.207774 14049 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260809 01:46:26.209066 14049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:46:26.212179 14049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:46:26.213626 14049 hybrid_clock.cc:648] HybridClock initialized: now 1786239986213587 us; error 54 us; skew 500 ppm
I20260809 01:46:26.214457 14049 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:46:26.221149 14049 webserver.cc:533] Webserver started at http://127.10.8.125:43683/ using document root <none> and password file <none>
I20260809 01:46:26.222152 14049 fs_manager.cc:362] Metadata directory not provided
I20260809 01:46:26.222415 14049 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:46:26.222898 14049 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:46:26.227269 14049 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/data/instance:
uuid: "3d2c2e2b6aa54060a87795a74d36e4d4"
format_stamp: "Formatted at 2026-08-09 01:46:26 on dist-test-slave-zr1t"
I20260809 01:46:26.228379 14049 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/wal/instance:
uuid: "3d2c2e2b6aa54060a87795a74d36e4d4"
format_stamp: "Formatted at 2026-08-09 01:46:26 on dist-test-slave-zr1t"
I20260809 01:46:26.235632 14049 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.005s sys 0.004s
I20260809 01:46:26.241457 14080 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:46:26.242604 14049 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260809 01:46:26.242950 14049 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/data,/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/wal
uuid: "3d2c2e2b6aa54060a87795a74d36e4d4"
format_stamp: "Formatted at 2026-08-09 01:46:26 on dist-test-slave-zr1t"
I20260809 01:46:26.243309 14049 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:46:26.296423 14049 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 01:46:26.298050 14049 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:46:26.298517 14049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:46:26.372376 14049 rpc_server.cc:307] RPC server started. Bound to: 127.10.8.125:43535
I20260809 01:46:26.372453 14152 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.8.125:43535 every 8 connection(s)
I20260809 01:46:26.375216 14049 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/data/info.pb
I20260809 01:46:26.381361 14153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:26.384897 10273 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu as pid 14049
I20260809 01:46:26.385308 10273 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-1/wal/instance
I20260809 01:46:26.387673 10273 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
/tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.124:34129
--webserver_interface=127.10.8.124
--webserver_port=0
--builtin_ntp_servers=127.10.8.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--hive_metastore_uris=thrift://127.0.0.1:35729
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260809 01:46:26.391826 14153 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:46:26.413873 14153 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:46:26.427992 14153 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } has no permanent_uuid. Determining permanent_uuid...
W20260809 01:46:26.430550 14081 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
W20260809 01:46:26.434715 14153 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:34129: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:26.484303 14153 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } attempt: 1
W20260809 01:46:26.488610 14153 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:34129: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:26.557020 14153 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } attempt: 2
W20260809 01:46:26.561429 14153 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:34129: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:26.652849 14153 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } attempt: 3
W20260809 01:46:26.657049 14153 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:34129: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
W20260809 01:46:26.716251 14157 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:46:26.716903 14157 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:46:26.717377 14157 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 01:46:26.750327 14157 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260809 01:46:26.750633 14157 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260809 01:46:26.750893 14157 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:46:26.751143 14157 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:46:26.751401 14157 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:46:26.751658 14157 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:46:26.788970 14157 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.8.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:35729
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.124:34129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.8.124
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 01:43:22 UTC on e7f111948823
build id 13251
TSAN enabled
I20260809 01:46:26.790385 14157 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:46:26.792068 14157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 01:46:26.800441 14153 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } attempt: 4
W20260809 01:46:26.802959 14168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:26.808239 14153 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:34129: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:26.902066 14047 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } attempt: 7
I20260809 01:46:26.933633 14047 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } has no permanent_uuid. Determining permanent_uuid...
W20260809 01:46:26.954267 14047 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:34129: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:27.013957 14047 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } attempt: 1
W20260809 01:46:27.021564 14047 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:34129: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:27.103158 14047 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } attempt: 2
I20260809 01:46:27.107709 14153 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } attempt: 5
W20260809 01:46:27.115232 14153 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:34129: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
W20260809 01:46:27.117805 14047 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:34129: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:27.205621 14047 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } attempt: 3
W20260809 01:46:27.215693 14047 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:34129: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:27.356240 14047 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } attempt: 4
W20260809 01:46:27.365404 14047 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:34129: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:27.648973 14047 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } attempt: 5
W20260809 01:46:27.656558 14047 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:34129: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:27.662755 14153 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } attempt: 6
W20260809 01:46:27.693563 14153 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:34129: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:28.209071 14047 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } attempt: 6
W20260809 01:46:28.241580 14047 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.8.124:34129: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
W20260809 01:46:28.208549 14166 debug-util.cc:398] Leaking SignalData structure 0x7b080002ee60 after lost signal to thread 14157
W20260809 01:46:28.534490 14157 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.731s user 0.614s sys 1.110s
W20260809 01:46:26.803609 14169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:28.534946 14157 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.732s user 0.614s sys 1.110s
W20260809 01:46:28.536841 14171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:28.543291 14170 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1737 milliseconds
I20260809 01:46:28.543391 14157 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260809 01:46:28.544879 14157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:46:28.547812 14157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:46:28.549275 14157 hybrid_clock.cc:648] HybridClock initialized: now 1786239988549197 us; error 92 us; skew 500 ppm
I20260809 01:46:28.550348 14157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:46:28.556725 14157 webserver.cc:533] Webserver started at http://127.10.8.124:44759/ using document root <none> and password file <none>
I20260809 01:46:28.557741 14157 fs_manager.cc:362] Metadata directory not provided
I20260809 01:46:28.557956 14157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:46:28.558360 14157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:46:28.562551 14157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data/instance:
uuid: "01fd3efb56f649edb893c33a263a0a93"
format_stamp: "Formatted at 2026-08-09 01:46:28 on dist-test-slave-zr1t"
I20260809 01:46:28.563582 14157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal/instance:
uuid: "01fd3efb56f649edb893c33a263a0a93"
format_stamp: "Formatted at 2026-08-09 01:46:28 on dist-test-slave-zr1t"
I20260809 01:46:28.571152 14157 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.000s sys 0.008s
I20260809 01:46:28.576553 14191 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:46:28.577708 14157 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260809 01:46:28.578083 14157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data,/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal
uuid: "01fd3efb56f649edb893c33a263a0a93"
format_stamp: "Formatted at 2026-08-09 01:46:28 on dist-test-slave-zr1t"
I20260809 01:46:28.578442 14157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:46:28.631742 14157 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 01:46:28.633095 14157 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:46:28.633508 14157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:46:28.708156 14157 rpc_server.cc:307] RPC server started. Bound to: 127.10.8.124:34129
I20260809 01:46:28.708267 14260 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.8.124:34129 every 8 connection(s)
I20260809 01:46:28.710915 14157 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data/info.pb
I20260809 01:46:28.714491 10273 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu as pid 14157
I20260809 01:46:28.714962 10273 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal/instance
I20260809 01:46:28.716699 14261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:28.735302 14261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:46:28.754127 14153 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } attempt: 7
I20260809 01:46:28.758520 14261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:46:28.770480 14261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:46:28.777655 14153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4: Bootstrap starting.
I20260809 01:46:28.784875 14153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:28.787271 14153 log.cc:826] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4: Log is configured to *not* fsync() on all Append() calls
I20260809 01:46:28.789762 14261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: Bootstrap starting.
I20260809 01:46:28.792397 14153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4: No bootstrap required, opened a new log
I20260809 01:46:28.797874 14261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:28.800129 14261 log.cc:826] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: Log is configured to *not* fsync() on all Append() calls
I20260809 01:46:28.805305 14261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: No bootstrap required, opened a new log
I20260809 01:46:28.820065 14153 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:28.820861 14153 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:28.821179 14153 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d2c2e2b6aa54060a87795a74d36e4d4, State: Initialized, Role: FOLLOWER
I20260809 01:46:28.822080 14153 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:28.824793 14279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:28.825631 14279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:28.826742 14153 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:46:28.833607 14261 raft_consensus.cc:359] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:28.834575 14261 raft_consensus.cc:385] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:28.834931 14261 raft_consensus.cc:740] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01fd3efb56f649edb893c33a263a0a93, State: Initialized, Role: FOLLOWER
I20260809 01:46:28.835958 14261 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:28.839012 14289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:28.839993 14289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:28.841269 14261 sys_catalog.cc:565] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 01:46:28.874500 14307 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 01:46:28.874958 14307 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 01:46:28.876415 14279 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:28.876926 14279 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:28.881385 14228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01fd3efb56f649edb893c33a263a0a93" is_pre_election: true
I20260809 01:46:28.882385 14228 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d2c2e2b6aa54060a87795a74d36e4d4 in term 0.
I20260809 01:46:28.883162 14279 leader_election.cc:290] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
I20260809 01:46:28.884073 14081 leader_election.cc:304] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93, 3d2c2e2b6aa54060a87795a74d36e4d4; no voters:
I20260809 01:46:28.885061 14279 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:28.885466 14279 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:28.885896 14279 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:46:28.889314 14197 tablet.cc:2406] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:46:28.892705 14279 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
W20260809 01:46:28.906756 14307 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 01:46:28.908345 14307 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 01:46:28.909905 14279 leader_election.cc:290] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [CANDIDATE]: Term 1 election: Requested vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
I20260809 01:46:28.910264 14228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01fd3efb56f649edb893c33a263a0a93"
I20260809 01:46:28.910821 14228 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:28.920974 14228 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d2c2e2b6aa54060a87795a74d36e4d4 in term 1.
I20260809 01:46:28.922854 14081 leader_election.cc:304] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93, 3d2c2e2b6aa54060a87795a74d36e4d4; no voters:
I20260809 01:46:28.923735 14279 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 1 FOLLOWER]: Leader election won for term 1
W20260809 01:46:28.923875 14307 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 01:46:28.924202 14307 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 01:46:28.939242 14279 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 1 LEADER]: Becoming Leader. State: Replica: 3d2c2e2b6aa54060a87795a74d36e4d4, State: Running, Role: LEADER
I20260809 01:46:28.944602 14279 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:28.967121 14326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d2c2e2b6aa54060a87795a74d36e4d4. Latest consensus state: current_term: 1 leader_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:28.967950 14326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: This master's current role is: LEADER
W20260809 01:46:28.974242 14307 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 01:46:28.974525 14307 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 01:46:28.980527 14334 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 01:46:29.012269 14334 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 01:46:29.024318 14339 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 01:46:28.867268 14049 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: OK, instance_id { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" instance_seqno: 1786239984521370 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 01:46:29.030736 14049 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:46:29.049266 14339 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 01:46:29.049896 14339 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 01:46:29.056524 14228 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Refusing update from remote peer 3d2c2e2b6aa54060a87795a74d36e4d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 01:46:29.058701 14087 tablet.cc:2406] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:46:29.061880 14326 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 01:46:29.064415 14339 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 01:46:29.064728 14339 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 01:46:29.037140 14157 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: OK, instance_id { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" instance_seqno: 1786239984521370 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 01:46:29.082566 14157 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:46:29.105064 14339 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 01:46:29.107429 14289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d2c2e2b6aa54060a87795a74d36e4d4. Latest consensus state: current_term: 1 leader_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:29.108119 14289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 01:46:29.105381 14339 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 01:46:29.108249 14340 mvcc.cc:204] Tried to move back new op lower bound from 7316438995109949440 to 7316438994729619456. Current Snapshot: MvccSnapshot[applied={T|T < 7316438995109949440}]
I20260809 01:46:29.115303 14289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:29.121601 14289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 01:46:29.123483 14339 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 01:46:29.123775 14339 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 01:46:29.125854 14279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:29.126559 14279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: This master's current role is: LEADER
I20260809 01:46:29.130213 14334 catalog_manager.cc:1383] Generated new cluster ID: a317462afe1741b5bf318d3556a05e8e
I20260809 01:46:29.130503 14334 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 01:46:29.142542 14339 catalog_manager.cc:1295] Loaded cluster ID: a317462afe1741b5bf318d3556a05e8e
I20260809 01:46:29.143286 14339 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: loading cluster ID for follower catalog manager: success
W20260809 01:46:29.158092 14339 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260809 01:46:29.159714 14339 catalog_manager.cc:883] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260809 01:46:29.164109 14334 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 01:46:29.166157 14334 catalog_manager.cc:1540] Loading token signing keys...
I20260809 01:46:29.178937 14339 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: acquiring CA information for follower catalog manager: success
W20260809 01:46:29.183408 14339 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
W20260809 01:46:29.205161 14339 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260809 01:46:29.207270 14334 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4: Generated new TSK 0
I20260809 01:46:29.210592 14334 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 01:46:29.221338 14339 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 01:46:29.222117 14334 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260809 01:46:29.230814 14334 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260809 01:46:29.259227 10273 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
/tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.8.65:0
--local_ip_for_outbound_sockets=127.10.8.65
--webserver_interface=127.10.8.65
--webserver_port=0
--tserver_master_addrs=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--builtin_ntp_servers=127.10.8.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260809 01:46:29.275082 14047 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } attempt: 7
I20260809 01:46:29.320909 14047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31: Bootstrap starting.
I20260809 01:46:29.335831 14047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:29.344440 14047 log.cc:826] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31: Log is configured to *not* fsync() on all Append() calls
I20260809 01:46:29.375221 14047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31: No bootstrap required, opened a new log
I20260809 01:46:29.404376 14047 raft_consensus.cc:359] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:29.405215 14047 raft_consensus.cc:385] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:29.405503 14047 raft_consensus.cc:740] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdec945a08924d01a5c58babffef9e31, State: Initialized, Role: FOLLOWER
I20260809 01:46:29.406585 14047 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:29.432245 14047 sys_catalog.cc:565] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:46:29.432411 14006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdec945a08924d01a5c58babffef9e31"
I20260809 01:46:29.433194 14006 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:29.436385 14008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdec945a08924d01a5c58babffef9e31" is_pre_election: true
W20260809 01:46:29.437355 14008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39734 (request call id 2) took 554 ms (client timeout 738 ms). Trace:
W20260809 01:46:29.437712 14008 rpcz_store.cc:269] 0809 01:46:28.883507 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:28.884041 (+ 534us) service_pool.cc:224] Handling call
0809 01:46:29.437285 (+553244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:46:29.444916 14006 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d2c2e2b6aa54060a87795a74d36e4d4 in term 1.
I20260809 01:46:29.446247 14004 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Refusing update from remote peer 3d2c2e2b6aa54060a87795a74d36e4d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 01:46:29.454525 14006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39734 (request call id 3) took 540 ms (client timeout 516 ms). Trace:
W20260809 01:46:29.454926 14006 rpcz_store.cc:269] 0809 01:46:28.913996 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 01:46:28.914172 (+ 176us) service_pool.cc:224] Handling call
0809 01:46:29.454464 (+540292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260809 01:46:29.430133 14353 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:29.459982 14353 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:29.463059 14006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdec945a08924d01a5c58babffef9e31"
I20260809 01:46:29.464336 14326 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:46:29.466243 14006 raft_consensus.cc:2376] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d2c2e2b6aa54060a87795a74d36e4d4 in term 1. Re-sending same reply.
W20260809 01:46:29.477559 13980 tablet.cc:2406] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:46:29.565707 14353 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d2c2e2b6aa54060a87795a74d36e4d4. Latest consensus state: current_term: 1 leader_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:29.566110 14367 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:29.566617 14353 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:29.566874 14367 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:29.588681 14381 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 01:46:29.618583 14381 catalog_manager.cc:1295] Loaded cluster ID: a317462afe1741b5bf318d3556a05e8e
I20260809 01:46:29.618866 14381 catalog_manager.cc:1588] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31: loading cluster ID for follower catalog manager: success
I20260809 01:46:29.633466 14381 catalog_manager.cc:1610] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31: acquiring CA information for follower catalog manager: success
I20260809 01:46:29.646018 14381 catalog_manager.cc:1638] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 01:46:29.718207 14384 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:29.718820 14384 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:29.727332 14004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cdec945a08924d01a5c58babffef9e31" is_pre_election: true
I20260809 01:46:29.727538 14384 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
I20260809 01:46:29.750861 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:29.752792 14195 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:46:29.753736 14384 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:46:29.975896 14326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:29.976753 14326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: This master's current role is: LEADER
W20260809 01:46:30.119725 14350 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:46:30.120458 14350 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:46:30.121393 14350 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 01:46:30.186789 14350 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:46:30.188494 14350 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.8.65
I20260809 01:46:30.259577 14350 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.8.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.8.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.8.65
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.8.65
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 01:43:22 UTC on e7f111948823
build id 13251
TSAN enabled
I20260809 01:46:30.261459 14350 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:46:30.263684 14350 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 01:46:30.302224 14395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:46:30.320478 14403 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:30.321583 14403 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:30.325996 14004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cdec945a08924d01a5c58babffef9e31" is_pre_election: true
I20260809 01:46:30.331609 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:30.333132 14195 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:46:30.344913 14403 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
I20260809 01:46:30.347646 14403 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:46:30.918406 14409 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:30.919350 14409 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:30.923974 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:30.926777 14409 leader_election.cc:290] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
I20260809 01:46:30.927857 14224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01fd3efb56f649edb893c33a263a0a93" is_pre_election: true
I20260809 01:46:30.929621 13972 leader_election.cc:304] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdec945a08924d01a5c58babffef9e31; no voters: 01fd3efb56f649edb893c33a263a0a93, 3d2c2e2b6aa54060a87795a74d36e4d4
I20260809 01:46:30.930352 14409 raft_consensus.cc:2749] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:46:31.796726 14410 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:31.797467 14410 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:31.810667 14004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cdec945a08924d01a5c58babffef9e31" is_pre_election: true
I20260809 01:46:31.815039 14410 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
I20260809 01:46:31.816210 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:31.818173 14195 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:46:31.819232 14410 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:46:32.007575 14411 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:32.009761 14411 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:32.013073 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:32.018549 14411 leader_election.cc:290] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
I20260809 01:46:32.019670 14224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01fd3efb56f649edb893c33a263a0a93" is_pre_election: true
I20260809 01:46:32.021432 13972 leader_election.cc:304] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdec945a08924d01a5c58babffef9e31; no voters: 01fd3efb56f649edb893c33a263a0a93, 3d2c2e2b6aa54060a87795a74d36e4d4
I20260809 01:46:32.026175 14411 raft_consensus.cc:2749] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 01:46:30.306169 14397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:31.686982 14393 debug-util.cc:398] Leaking SignalData structure 0x7b0800037500 after lost signal to thread 14350
W20260809 01:46:32.093331 14399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:32.094286 14393 kernel_stack_watchdog.cc:198] Thread 14350 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 382ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260809 01:46:32.093596 14398 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1787 milliseconds
W20260809 01:46:32.095254 14350 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.792s user 0.247s sys 0.492s
W20260809 01:46:32.095569 14350 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.793s user 0.247s sys 0.492s
I20260809 01:46:32.095853 14350 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260809 01:46:32.100412 14350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:46:32.116842 14350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:46:32.118294 14350 hybrid_clock.cc:648] HybridClock initialized: now 1786239992118246 us; error 47 us; skew 500 ppm
I20260809 01:46:32.119375 14350 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:46:32.142861 14350 webserver.cc:533] Webserver started at http://127.10.8.65:33199/ using document root <none> and password file <none>
I20260809 01:46:32.148348 14350 fs_manager.cc:362] Metadata directory not provided
I20260809 01:46:32.148675 14350 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:46:32.149264 14350 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:46:32.156334 14350 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/data/instance:
uuid: "9336b0d6675b4225831595357c4bdc3b"
format_stamp: "Formatted at 2026-08-09 01:46:32 on dist-test-slave-zr1t"
I20260809 01:46:32.157871 14350 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/wal/instance:
uuid: "9336b0d6675b4225831595357c4bdc3b"
format_stamp: "Formatted at 2026-08-09 01:46:32 on dist-test-slave-zr1t"
I20260809 01:46:32.182192 14350 fs_manager.cc:696] Time spent creating directory manager: real 0.023s user 0.008s sys 0.011s
I20260809 01:46:32.202572 14418 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:46:32.204504 14350 fs_manager.cc:730] Time spent opening block manager: real 0.017s user 0.014s sys 0.002s
I20260809 01:46:32.205075 14350 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/data,/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/wal
uuid: "9336b0d6675b4225831595357c4bdc3b"
format_stamp: "Formatted at 2026-08-09 01:46:32 on dist-test-slave-zr1t"
I20260809 01:46:32.206645 14350 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:46:32.349858 14350 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 01:46:32.352080 14350 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:46:32.352770 14350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:46:32.363157 14350 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 01:46:32.375633 14350 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:46:32.376119 14350 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260809 01:46:32.377179 14350 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 01:46:32.377499 14350 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:46:32.568439 14468 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:32.568923 14468 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:32.570905 14468 leader_election.cc:290] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
I20260809 01:46:32.571748 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:32.572368 14224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01fd3efb56f649edb893c33a263a0a93" is_pre_election: true
I20260809 01:46:32.574011 13972 leader_election.cc:304] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdec945a08924d01a5c58babffef9e31; no voters: 01fd3efb56f649edb893c33a263a0a93, 3d2c2e2b6aa54060a87795a74d36e4d4
I20260809 01:46:32.588301 14468 raft_consensus.cc:2749] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:46:32.743635 14500 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:32.744127 14500 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:32.749920 14004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cdec945a08924d01a5c58babffef9e31" is_pre_election: true
I20260809 01:46:32.762481 14500 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
I20260809 01:46:32.763065 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:32.764756 14195 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:46:32.765734 14500 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:46:32.893922 14350 rpc_server.cc:307] RPC server started. Bound to: 127.10.8.65:41981
I20260809 01:46:32.894071 14589 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.8.65:41981 every 8 connection(s)
I20260809 01:46:32.897511 14350 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/data/info.pb
I20260809 01:46:32.903425 10273 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu as pid 14350
I20260809 01:46:32.904086 10273 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/wal/instance
I20260809 01:46:32.920773 10273 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
/tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.8.66:0
--local_ip_for_outbound_sockets=127.10.8.66
--webserver_interface=127.10.8.66
--webserver_port=0
--tserver_master_addrs=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--builtin_ntp_servers=127.10.8.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260809 01:46:33.038250 14592 heartbeater.cc:344] Connected to a master server at 127.10.8.124:34129
I20260809 01:46:33.038709 14592 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:33.040050 14592 heartbeater.cc:507] Master 127.10.8.124:34129 requested a full tablet report, sending...
I20260809 01:46:33.047317 14591 heartbeater.cc:344] Connected to a master server at 127.10.8.125:43535
I20260809 01:46:33.047659 14591 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:33.051427 14210 ts_manager.cc:194] Registered new tserver with Master: 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981)
I20260809 01:46:33.049484 14591 heartbeater.cc:507] Master 127.10.8.125:43535 requested a full tablet report, sending...
I20260809 01:46:33.055521 14103 ts_manager.cc:194] Registered new tserver with Master: 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981)
I20260809 01:46:33.058946 14103 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.10.8.65:35481
I20260809 01:46:33.092895 14597 heartbeater.cc:344] Connected to a master server at 127.10.8.126:43137
I20260809 01:46:33.093281 14597 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:33.093986 14597 heartbeater.cc:507] Master 127.10.8.126:43137 requested a full tablet report, sending...
I20260809 01:46:33.096948 13991 ts_manager.cc:194] Registered new tserver with Master: 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981)
I20260809 01:46:33.117666 14610 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:33.118291 14610 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:33.120343 14610 leader_election.cc:290] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
I20260809 01:46:33.132623 14224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01fd3efb56f649edb893c33a263a0a93" is_pre_election: true
I20260809 01:46:33.140276 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:33.142318 13975 leader_election.cc:304] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdec945a08924d01a5c58babffef9e31; no voters: 01fd3efb56f649edb893c33a263a0a93, 3d2c2e2b6aa54060a87795a74d36e4d4
I20260809 01:46:33.147574 14610 raft_consensus.cc:2749] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 01:46:33.545995 14596 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:46:33.546651 14596 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:46:33.547525 14596 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 01:46:33.563311 14591 heartbeater.cc:499] Master 127.10.8.125:43535 was elected leader, sending a full tablet report...
W20260809 01:46:33.611150 14596 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:46:33.612663 14596 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.8.66
I20260809 01:46:33.679636 14596 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.8.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.8.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.8.66
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.8.66
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 01:43:22 UTC on e7f111948823
build id 13251
TSAN enabled
I20260809 01:46:33.681237 14596 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:46:33.683434 14596 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 01:46:33.699146 14618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:46:33.742586 14625 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:33.748546 14625 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:33.753255 14004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cdec945a08924d01a5c58babffef9e31" is_pre_election: true
I20260809 01:46:33.757812 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:33.759907 14195 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:46:33.767307 14625 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
I20260809 01:46:33.773101 14625 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:46:34.780009 14629 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:34.780606 14629 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:34.787254 14004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cdec945a08924d01a5c58babffef9e31" is_pre_election: true
I20260809 01:46:34.803108 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:34.807945 14195 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:46:34.811591 14629 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
I20260809 01:46:34.812176 14629 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 01:46:35.106019 14617 debug-util.cc:398] Leaking SignalData structure 0x7b0800037500 after lost signal to thread 14596
W20260809 01:46:35.296911 14617 kernel_stack_watchdog.cc:198] Thread 14596 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 399ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260809 01:46:33.699849 14619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:35.302973 14596 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.602s user 0.000s sys 0.005s
W20260809 01:46:35.303269 14620 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1598 milliseconds
W20260809 01:46:35.303369 14596 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.603s user 0.000s sys 0.005s
W20260809 01:46:35.304657 14622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:46:35.304719 14596 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260809 01:46:35.306301 14596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:46:35.308987 14596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:46:35.310433 14596 hybrid_clock.cc:648] HybridClock initialized: now 1786239995310390 us; error 45 us; skew 500 ppm
I20260809 01:46:35.311178 14596 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:46:35.317829 14596 webserver.cc:533] Webserver started at http://127.10.8.66:43227/ using document root <none> and password file <none>
I20260809 01:46:35.318728 14596 fs_manager.cc:362] Metadata directory not provided
I20260809 01:46:35.318912 14596 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:46:35.319315 14596 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:46:35.324195 14596 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/data/instance:
uuid: "36abc3bb1604427cb977fa2e6a6f4272"
format_stamp: "Formatted at 2026-08-09 01:46:35 on dist-test-slave-zr1t"
I20260809 01:46:35.325297 14596 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/wal/instance:
uuid: "36abc3bb1604427cb977fa2e6a6f4272"
format_stamp: "Formatted at 2026-08-09 01:46:35 on dist-test-slave-zr1t"
I20260809 01:46:35.333091 14596 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.002s sys 0.004s
I20260809 01:46:35.339300 14639 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:46:35.340369 14596 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260809 01:46:35.340708 14596 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/data,/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/wal
uuid: "36abc3bb1604427cb977fa2e6a6f4272"
format_stamp: "Formatted at 2026-08-09 01:46:35 on dist-test-slave-zr1t"
I20260809 01:46:35.341038 14596 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:46:35.407375 14596 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 01:46:35.408860 14596 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:46:35.409329 14596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:46:35.412160 14596 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 01:46:35.416961 14596 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:46:35.417158 14596 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:46:35.417358 14596 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 01:46:35.417490 14596 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:46:35.574194 14596 rpc_server.cc:307] RPC server started. Bound to: 127.10.8.66:40383
I20260809 01:46:35.574364 14828 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.8.66:40383 every 8 connection(s)
I20260809 01:46:35.577217 14596 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/data/info.pb
I20260809 01:46:35.584951 10273 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu as pid 14596
I20260809 01:46:35.585655 10273 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/wal/instance
I20260809 01:46:35.604521 10273 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
/tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.8.67:0
--local_ip_for_outbound_sockets=127.10.8.67
--webserver_interface=127.10.8.67
--webserver_port=0
--tserver_master_addrs=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--builtin_ntp_servers=127.10.8.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260809 01:46:35.636970 14831 heartbeater.cc:344] Connected to a master server at 127.10.8.124:34129
I20260809 01:46:35.637563 14831 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:35.637791 14830 heartbeater.cc:344] Connected to a master server at 127.10.8.125:43535
I20260809 01:46:35.638091 14832 heartbeater.cc:344] Connected to a master server at 127.10.8.126:43137
I20260809 01:46:35.638522 14832 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:35.638962 14831 heartbeater.cc:507] Master 127.10.8.124:34129 requested a full tablet report, sending...
I20260809 01:46:35.639793 14832 heartbeater.cc:507] Master 127.10.8.126:43137 requested a full tablet report, sending...
I20260809 01:46:35.638142 14830 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:35.641172 14830 heartbeater.cc:507] Master 127.10.8.125:43535 requested a full tablet report, sending...
I20260809 01:46:35.642216 14210 ts_manager.cc:194] Registered new tserver with Master: 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383)
I20260809 01:46:35.642980 13991 ts_manager.cc:194] Registered new tserver with Master: 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383)
I20260809 01:46:35.644796 14103 ts_manager.cc:194] Registered new tserver with Master: 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383)
I20260809 01:46:35.646247 14103 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.10.8.66:39359
I20260809 01:46:35.897262 14849 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:35.897789 14849 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:35.901283 14849 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
I20260809 01:46:35.901778 14004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cdec945a08924d01a5c58babffef9e31" is_pre_election: true
I20260809 01:46:35.902133 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:35.904042 14195 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:46:35.904805 14849 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:46:35.950889 14856 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:35.951332 14856 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:35.952966 14856 leader_election.cc:290] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
I20260809 01:46:35.953650 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:35.954041 14224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01fd3efb56f649edb893c33a263a0a93" is_pre_election: true
I20260809 01:46:35.955567 13972 leader_election.cc:304] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdec945a08924d01a5c58babffef9e31; no voters: 01fd3efb56f649edb893c33a263a0a93, 3d2c2e2b6aa54060a87795a74d36e4d4
I20260809 01:46:35.956157 14856 raft_consensus.cc:2749] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 01:46:35.961151 14847 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:46:35.961702 14847 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:46:35.962298 14847 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 01:46:35.996341 14847 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:46:35.997244 14847 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.8.67
I20260809 01:46:36.032476 14847 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.8.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.8.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.8.67
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.8.67
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 01:43:22 UTC on e7f111948823
build id 13251
TSAN enabled
I20260809 01:46:36.033990 14847 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:46:36.035615 14847 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 01:46:36.048898 14860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:46:36.149489 14830 heartbeater.cc:499] Master 127.10.8.125:43535 was elected leader, sending a full tablet report...
I20260809 01:46:36.445165 14868 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:36.446388 14868 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:36.450768 14004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cdec945a08924d01a5c58babffef9e31" is_pre_election: true
I20260809 01:46:36.450609 14868 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
I20260809 01:46:36.453857 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:36.459627 14193 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:46:36.460681 14868 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:46:36.960351 14871 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:36.961067 14871 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:36.965565 14871 leader_election.cc:290] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
I20260809 01:46:36.966495 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:36.968652 14224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01fd3efb56f649edb893c33a263a0a93" is_pre_election: true
I20260809 01:46:36.971529 13972 leader_election.cc:304] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdec945a08924d01a5c58babffef9e31; no voters: 01fd3efb56f649edb893c33a263a0a93, 3d2c2e2b6aa54060a87795a74d36e4d4
I20260809 01:46:36.972608 14871 raft_consensus.cc:2749] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:46:37.009146 14872 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:37.009925 14872 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:37.014221 14004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cdec945a08924d01a5c58babffef9e31" is_pre_election: true
I20260809 01:46:37.014822 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:37.015034 14004 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01fd3efb56f649edb893c33a263a0a93 in term 1.
I20260809 01:46:37.017089 14872 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
I20260809 01:46:37.018114 14193 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93, cdec945a08924d01a5c58babffef9e31; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4
I20260809 01:46:37.019793 14872 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 01:46:37.020351 14872 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:37.020987 14872 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 1 FOLLOWER]: Advancing to term 2
I20260809 01:46:37.035779 14872 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:37.040010 14004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cdec945a08924d01a5c58babffef9e31"
I20260809 01:46:37.040757 14004 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 1 FOLLOWER]: Advancing to term 2
I20260809 01:46:37.041198 14224 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 2 FOLLOWER]: Rejecting Update request from peer 3d2c2e2b6aa54060a87795a74d36e4d4 for earlier term 1. Current term is 2. Ops: []
I20260809 01:46:37.060719 14873 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 01:46:37.070483 14004 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01fd3efb56f649edb893c33a263a0a93 in term 2.
I20260809 01:46:37.074288 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01fd3efb56f649edb893c33a263a0a93" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4"
I20260809 01:46:37.077230 14872 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 election: Requested vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
I20260809 01:46:37.084244 14193 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93, cdec945a08924d01a5c58babffef9e31; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4
I20260809 01:46:37.086022 14872 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 01:46:37.093284 14875 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 01:46:37.093873 14875 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d2c2e2b6aa54060a87795a74d36e4d4, State: Running, Role: LEADER
I20260809 01:46:37.094117 14872 raft_consensus.cc:697] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 2 LEADER]: Becoming Leader. State: Replica: 01fd3efb56f649edb893c33a263a0a93, State: Running, Role: LEADER
I20260809 01:46:37.095093 14875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:37.096037 14872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:37.104525 14004 raft_consensus.cc:1240] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 2 FOLLOWER]: Rejecting Update request from peer 3d2c2e2b6aa54060a87795a74d36e4d4 for earlier term 1. Current term is 2. Ops: []
I20260809 01:46:37.105959 14875 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 01:46:37.115620 14876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01fd3efb56f649edb893c33a263a0a93. Latest consensus state: current_term: 2 leader_uuid: "01fd3efb56f649edb893c33a263a0a93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:37.120325 14876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: This master's current role is: LEADER
I20260809 01:46:37.127694 14880 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 01:46:37.144351 14880 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 01:46:37.146737 14307 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 01:46:37.149931 14880 catalog_manager.cc:1295] Loaded cluster ID: a317462afe1741b5bf318d3556a05e8e
I20260809 01:46:37.150249 14880 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 01:46:37.160355 14880 catalog_manager.cc:1540] Loading token signing keys...
I20260809 01:46:37.165254 14880 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: Loaded TSK: 0
I20260809 01:46:37.166651 14880 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 01:46:37.171314 14880 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260809 01:46:37.175287 14880 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
W20260809 01:46:36.050180 14861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:37.361812 14862 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
W20260809 01:46:37.363672 14864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:37.366804 14847 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.317s user 0.427s sys 0.861s
W20260809 01:46:37.367136 14847 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.318s user 0.427s sys 0.861s
I20260809 01:46:37.367439 14847 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260809 01:46:37.369058 14847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:46:37.372045 14847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:46:37.373538 14847 hybrid_clock.cc:648] HybridClock initialized: now 1786239997373434 us; error 69 us; skew 500 ppm
I20260809 01:46:37.375034 14847 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:46:37.384583 14847 webserver.cc:533] Webserver started at http://127.10.8.67:36969/ using document root <none> and password file <none>
I20260809 01:46:37.385926 14847 fs_manager.cc:362] Metadata directory not provided
I20260809 01:46:37.386245 14847 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:46:37.386893 14847 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:46:37.392541 14847 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/data/instance:
uuid: "e3d5dac2381748eabc723a0c471ad03f"
format_stamp: "Formatted at 2026-08-09 01:46:37 on dist-test-slave-zr1t"
I20260809 01:46:37.393731 14847 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/wal/instance:
uuid: "e3d5dac2381748eabc723a0c471ad03f"
format_stamp: "Formatted at 2026-08-09 01:46:37 on dist-test-slave-zr1t"
I20260809 01:46:37.402259 14847 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.005s sys 0.002s
I20260809 01:46:37.409094 14885 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:46:37.410625 14847 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260809 01:46:37.410986 14847 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/data,/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/wal
uuid: "e3d5dac2381748eabc723a0c471ad03f"
format_stamp: "Formatted at 2026-08-09 01:46:37 on dist-test-slave-zr1t"
I20260809 01:46:37.411293 14847 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:46:37.478942 14847 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 01:46:37.480746 14847 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:46:37.481235 14847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:46:37.484539 14847 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 01:46:37.489226 14847 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:46:37.489532 14847 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:46:37.489833 14847 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 01:46:37.490062 14847 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:46:37.490559 14004 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 2 FOLLOWER]: Refusing update from remote peer 01fd3efb56f649edb893c33a263a0a93: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 01:46:37.492285 14876 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 01:46:37.497715 14899 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01fd3efb56f649edb893c33a263a0a93. Latest consensus state: current_term: 2 leader_uuid: "01fd3efb56f649edb893c33a263a0a93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:37.498381 14899 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:37.501780 14872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "01fd3efb56f649edb893c33a263a0a93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:37.504739 14121 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 2 FOLLOWER]: Refusing update from remote peer 01fd3efb56f649edb893c33a263a0a93: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 01:46:37.507320 14872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: This master's current role is: LEADER
I20260809 01:46:37.507995 14872 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 01:46:37.511962 14899 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "01fd3efb56f649edb893c33a263a0a93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:37.512571 14899 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:37.521792 14875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01fd3efb56f649edb893c33a263a0a93. Latest consensus state: current_term: 2 leader_uuid: "01fd3efb56f649edb893c33a263a0a93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:37.522326 14875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:37.522357 14874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "01fd3efb56f649edb893c33a263a0a93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:37.523061 14874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:37.667666 14847 rpc_server.cc:307] RPC server started. Bound to: 127.10.8.67:38177
I20260809 01:46:37.667802 15073 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.8.67:38177 every 8 connection(s)
I20260809 01:46:37.670501 14847 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/data/info.pb
I20260809 01:46:37.671115 10273 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu as pid 14847
I20260809 01:46:37.671530 10273 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/wal/instance
I20260809 01:46:37.737375 15074 heartbeater.cc:344] Connected to a master server at 127.10.8.125:43535
I20260809 01:46:37.737934 15074 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:37.739341 15074 heartbeater.cc:507] Master 127.10.8.125:43535 requested a full tablet report, sending...
I20260809 01:46:37.742339 14103 ts_manager.cc:194] Registered new tserver with Master: e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177)
I20260809 01:46:37.745319 15075 heartbeater.cc:344] Connected to a master server at 127.10.8.124:34129
I20260809 01:46:37.745599 15076 heartbeater.cc:344] Connected to a master server at 127.10.8.126:43137
I20260809 01:46:37.745998 15076 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:37.745661 15075 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:37.746891 15076 heartbeater.cc:507] Master 127.10.8.126:43137 requested a full tablet report, sending...
I20260809 01:46:37.747042 15075 heartbeater.cc:507] Master 127.10.8.124:34129 requested a full tablet report, sending...
I20260809 01:46:37.749828 14210 ts_manager.cc:194] Registered new tserver with Master: e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177)
I20260809 01:46:37.750349 13991 ts_manager.cc:194] Registered new tserver with Master: e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177)
I20260809 01:46:37.752439 14210 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.10.8.67:49953
I20260809 01:46:37.757992 10273 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 01:46:37.807853 14210 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55588:
name: "default.Table_75dfb3d85c0d413fb1d73c3ab0b38f18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:37.809252 14209 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55588:
name: "default.Table_409febef55ad4ddb8638e01e247c19fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:37.814229 14208 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55588:
name: "default.Table_645ec611a69b474199c9493389824d89"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:37.818991 14207 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55588:
name: "default.Table_ed71d957cdb34b9fa97483de90b1d539"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:37.823254 14210 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:46:37.827507 14208 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_645ec611a69b474199c9493389824d89 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:46:37.830991 14207 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_ed71d957cdb34b9fa97483de90b1d539 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:46:37.832752 14209 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_409febef55ad4ddb8638e01e247c19fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:46:37.846665 10273 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu with pid 14157
W20260809 01:46:37.907500 15094 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.8.124:34129)
I20260809 01:46:38.001415 14899 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01fd3efb56f649edb893c33a263a0a93)
W20260809 01:46:38.001791 15094 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.CreateTable -> {remote=127.10.8.124:34129, user_credentials={real_user=slave}} blocked reactor thread for 96306.8us
I20260809 01:46:38.001901 14899 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:38.004007 14899 leader_election.cc:290] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
I20260809 01:46:38.005756 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:38.012270 14874 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01fd3efb56f649edb893c33a263a0a93)
I20260809 01:46:38.012828 14874 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:38.015033 14874 leader_election.cc:290] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
I20260809 01:46:38.017239 14004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cdec945a08924d01a5c58babffef9e31" is_pre_election: true
I20260809 01:46:38.017884 14004 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d2c2e2b6aa54060a87795a74d36e4d4 in term 2.
I20260809 01:46:38.019120 14082 leader_election.cc:304] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31; no voters:
W20260809 01:46:38.019565 14081 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111) [suppressed 7 similar messages]
I20260809 01:46:38.020010 14874 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 01:46:38.020305 14874 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01fd3efb56f649edb893c33a263a0a93)
I20260809 01:46:38.020678 14874 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 2 FOLLOWER]: Advancing to term 3
I20260809 01:46:38.030246 14874 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:38.033457 14004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cdec945a08924d01a5c58babffef9e31"
I20260809 01:46:38.034088 14004 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 2 FOLLOWER]: Advancing to term 3
W20260809 01:46:38.039222 13972 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111) [suppressed 13 similar messages]
I20260809 01:46:38.045212 14004 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d2c2e2b6aa54060a87795a74d36e4d4 in term 3.
I20260809 01:46:38.047348 14874 leader_election.cc:290] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [CANDIDATE]: Term 3 election: Requested vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
I20260809 01:46:38.050467 14082 leader_election.cc:304] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31; no voters:
I20260809 01:46:38.051988 14874 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 01:46:38.062711 14874 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [term 3 LEADER]: Becoming Leader. State: Replica: 3d2c2e2b6aa54060a87795a74d36e4d4, State: Running, Role: LEADER
I20260809 01:46:38.064132 14874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
W20260809 01:46:38.066365 14081 leader_election.cc:336] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129): Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:38.072261 15147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d2c2e2b6aa54060a87795a74d36e4d4. Latest consensus state: current_term: 3 leader_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:38.072940 15147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: This master's current role is: LEADER
W20260809 01:46:38.074971 14081 leader_election.cc:336] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129): Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
W20260809 01:46:38.078073 13972 leader_election.cc:336] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129): Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:38.078714 13972 leader_election.cc:304] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdec945a08924d01a5c58babffef9e31; no voters: 01fd3efb56f649edb893c33a263a0a93, 3d2c2e2b6aa54060a87795a74d36e4d4
I20260809 01:46:38.079751 14899 raft_consensus.cc:2749] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 01:46:38.081598 15148 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 01:46:38.091463 15148 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 01:46:38.097800 15148 catalog_manager.cc:1295] Loaded cluster ID: a317462afe1741b5bf318d3556a05e8e
I20260809 01:46:38.098030 15148 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 01:46:38.101248 15148 catalog_manager.cc:1540] Loading token signing keys...
I20260809 01:46:38.105178 15148 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4: Loaded TSK: 0
I20260809 01:46:38.117064 15148 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 01:46:38.125113 14592 heartbeater.cc:499] Master 127.10.8.124:34129 was elected leader, sending a full tablet report...
I20260809 01:46:38.152954 15148 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
W20260809 01:46:38.153651 14592 heartbeater.cc:646] Failed to heartbeat to 127.10.8.124:34129 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:38.159345 15148 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260809 01:46:38.162176 14831 heartbeater.cc:499] Master 127.10.8.124:34129 was elected leader, sending a full tablet report...
I20260809 01:46:38.171427 14103 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33272:
name: "default.Table_ed71d957cdb34b9fa97483de90b1d539"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:38.176803 14100 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33272:
name: "default.Table_409febef55ad4ddb8638e01e247c19fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:38.178386 14098 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33272:
name: "default.Table_75dfb3d85c0d413fb1d73c3ab0b38f18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:38.180868 14101 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33272:
name: "default.Table_645ec611a69b474199c9493389824d89"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:46:38.183691 14103 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_ed71d957cdb34b9fa97483de90b1d539 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:46:38.186270 14100 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_409febef55ad4ddb8638e01e247c19fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:46:38.187680 14098 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:46:38.198855 14101 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_645ec611a69b474199c9493389824d89 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 01:46:38.210413 14831 heartbeater.cc:646] Failed to heartbeat to 127.10.8.124:34129 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:38.255944 15075 heartbeater.cc:499] Master 127.10.8.124:34129 was elected leader, sending a full tablet report...
W20260809 01:46:38.280633 15075 heartbeater.cc:646] Failed to heartbeat to 127.10.8.124:34129 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:38.456602 10273 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
/tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.124:34129
--webserver_interface=127.10.8.124
--webserver_port=44759
--builtin_ntp_servers=127.10.8.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--hive_metastore_uris=thrift://127.0.0.1:35729
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260809 01:46:38.509753 14101 catalog_manager.cc:2200] created HMS catalog entry for table [id=5ed9016f7a8843bdb0dab988c9968d68]
I20260809 01:46:38.518903 14004 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Refusing update from remote peer 3d2c2e2b6aa54060a87795a74d36e4d4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 01:46:38.520831 15147 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260809 01:46:38.551430 14081 consensus_peers.cc:597] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 -> Peer 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129): Couldn't send request to peer 01fd3efb56f649edb893c33a263a0a93. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 01:46:38.565109 14899 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d2c2e2b6aa54060a87795a74d36e4d4. Latest consensus state: current_term: 3 leader_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:38.565855 14899 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:38.595283 14098 catalog_manager.cc:2200] created HMS catalog entry for table [id=9ec275401ed84e09b74eb2b454e37c4f]
I20260809 01:46:38.603336 14899 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:38.603981 14899 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:38.621827 15147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:38.622761 15147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4 [sys.catalog]: This master's current role is: LEADER
I20260809 01:46:38.680492 14830 heartbeater.cc:499] Master 127.10.8.125:43535 was elected leader, sending a full tablet report...
I20260809 01:46:38.691418 14101 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33272:
table { table_name: "default.Table_645ec611a69b474199c9493389824d89" } new_table_name: "default.Table_1ee29cfe13ec46e388603a636a24eb32" modify_external_catalogs: true
I20260809 01:46:38.693068 14096 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33272:
table { table_name: "default.Table_75dfb3d85c0d413fb1d73c3ab0b38f18" } new_table_name: "default.Table_b9994acb9eb342888a41b725ad2f0c87" modify_external_catalogs: true
I20260809 01:46:38.748224 15074 heartbeater.cc:499] Master 127.10.8.125:43535 was elected leader, sending a full tablet report...
I20260809 01:46:38.775529 14969 tablet_service.cc:1511] Processing CreateTablet for tablet 1c76ea235a1a4621b2bb4962c6f94656 (DEFAULT_TABLE table=default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 [id=9ec275401ed84e09b74eb2b454e37c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:38.777633 14968 tablet_service.cc:1511] Processing CreateTablet for tablet e8174401df624fc8bb9fcd4e675042db (DEFAULT_TABLE table=default.table_645ec611a69b474199c9493389824d89 [id=5ed9016f7a8843bdb0dab988c9968d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:38.779237 14968 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8174401df624fc8bb9fcd4e675042db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:38.786381 14969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c76ea235a1a4621b2bb4962c6f94656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:38.815099 14496 tablet_service.cc:1511] Processing CreateTablet for tablet e8174401df624fc8bb9fcd4e675042db (DEFAULT_TABLE table=default.table_645ec611a69b474199c9493389824d89 [id=5ed9016f7a8843bdb0dab988c9968d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:38.818159 14718 tablet_service.cc:1511] Processing CreateTablet for tablet 1c76ea235a1a4621b2bb4962c6f94656 (DEFAULT_TABLE table=default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 [id=9ec275401ed84e09b74eb2b454e37c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:38.818871 14497 tablet_service.cc:1511] Processing CreateTablet for tablet 1c76ea235a1a4621b2bb4962c6f94656 (DEFAULT_TABLE table=default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 [id=9ec275401ed84e09b74eb2b454e37c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:38.819479 14717 tablet_service.cc:1511] Processing CreateTablet for tablet e8174401df624fc8bb9fcd4e675042db (DEFAULT_TABLE table=default.table_645ec611a69b474199c9493389824d89 [id=5ed9016f7a8843bdb0dab988c9968d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:38.820534 14496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8174401df624fc8bb9fcd4e675042db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:38.820973 14717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8174401df624fc8bb9fcd4e675042db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:38.826251 14718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c76ea235a1a4621b2bb4962c6f94656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:38.829655 14497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c76ea235a1a4621b2bb4962c6f94656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:38.870942 14098 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33272:
name: "default.Table_1a5bb6557e7a4ffcb31eb9508c96b920"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:38.886235 14097 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33272:
name: "default.Table_5fc49430cb3041e9bfed3d29e3168bb0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:38.915037 15184 tablet_bootstrap.cc:492] T e8174401df624fc8bb9fcd4e675042db P 9336b0d6675b4225831595357c4bdc3b: Bootstrap starting.
I20260809 01:46:38.919775 15181 tablet_bootstrap.cc:492] T e8174401df624fc8bb9fcd4e675042db P e3d5dac2381748eabc723a0c471ad03f: Bootstrap starting.
I20260809 01:46:38.923920 15185 tablet_bootstrap.cc:492] T 1c76ea235a1a4621b2bb4962c6f94656 P 36abc3bb1604427cb977fa2e6a6f4272: Bootstrap starting.
I20260809 01:46:38.943699 14095 master_service.cc:960] ReplaceTablet: received request to replace tablet 1c76ea235a1a4621b2bb4962c6f94656 from {username='slave'} at 127.0.0.1:33272
I20260809 01:46:38.944404 15184 tablet_bootstrap.cc:654] T e8174401df624fc8bb9fcd4e675042db P 9336b0d6675b4225831595357c4bdc3b: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:38.944731 14094 master_service.cc:960] ReplaceTablet: received request to replace tablet e8174401df624fc8bb9fcd4e675042db from {username='slave'} at 127.0.0.1:33272
01:46:38.948 [HMS - ERROR - pool-6-thread-1] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_409febef55ad4ddb8638e01e247c19fb already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at jdk.proxy2/jdk.proxy2.$Proxy27.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
I20260809 01:46:38.953047 15185 tablet_bootstrap.cc:654] T 1c76ea235a1a4621b2bb4962c6f94656 P 36abc3bb1604427cb977fa2e6a6f4272: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:38.953080 15181 tablet_bootstrap.cc:654] T e8174401df624fc8bb9fcd4e675042db P e3d5dac2381748eabc723a0c471ad03f: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:38.955536 15185 log.cc:826] T 1c76ea235a1a4621b2bb4962c6f94656 P 36abc3bb1604427cb977fa2e6a6f4272: Log is configured to *not* fsync() on all Append() calls
I20260809 01:46:38.957063 15184 log.cc:826] T e8174401df624fc8bb9fcd4e675042db P 9336b0d6675b4225831595357c4bdc3b: Log is configured to *not* fsync() on all Append() calls
I20260809 01:46:38.960186 15181 log.cc:826] T e8174401df624fc8bb9fcd4e675042db P e3d5dac2381748eabc723a0c471ad03f: Log is configured to *not* fsync() on all Append() calls
W20260809 01:46:38.982712 14100 catalog_manager.cc:2196] Already present: failed to create HMS catalog entry for table [id=821e5e9d8bea42f0b1df27dfd3a0d9b0]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_409febef55ad4ddb8638e01e247c19fb already exists)
I20260809 01:46:38.986958 14095 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4: Sending DeleteTablet for 3 replicas of tablet 1c76ea235a1a4621b2bb4962c6f94656
I20260809 01:46:38.987737 14100 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33272:
name: "default.Table_b0eef29d161f4a1e8b63b62f0e7954d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:38.991918 14718 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c76ea235a1a4621b2bb4962c6f94656 with delete_type TABLET_DATA_DELETED (replaced by tablet 27a27ed554414d60a026addeee969bd4) from {username='slave'} at 127.0.0.1:42244
I20260809 01:46:39.006227 14094 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4: Sending DeleteTablet for 3 replicas of tablet e8174401df624fc8bb9fcd4e675042db
I20260809 01:46:39.007927 15184 tablet_bootstrap.cc:492] T e8174401df624fc8bb9fcd4e675042db P 9336b0d6675b4225831595357c4bdc3b: No bootstrap required, opened a new log
I20260809 01:46:39.010017 15184 ts_tablet_manager.cc:1403] T e8174401df624fc8bb9fcd4e675042db P 9336b0d6675b4225831595357c4bdc3b: Time spent bootstrapping tablet: real 0.096s user 0.013s sys 0.017s
I20260809 01:46:39.009987 14968 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c76ea235a1a4621b2bb4962c6f94656 with delete_type TABLET_DATA_DELETED (replaced by tablet 27a27ed554414d60a026addeee969bd4) from {username='slave'} at 127.0.0.1:45636
W20260809 01:46:39.014210 14083 catalog_manager.cc:5030] TS 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383): delete failed for tablet 1c76ea235a1a4621b2bb4962c6f94656 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c76ea235a1a4621b2bb4962c6f94656 already in progress: creating tablet
I20260809 01:46:39.008132 14496 tablet_service.cc:1558] Processing DeleteTablet for tablet e8174401df624fc8bb9fcd4e675042db with delete_type TABLET_DATA_DELETED (replaced by tablet be44f2b9e9fa4ed9812b51e4aafddedf) from {username='slave'} at 127.0.0.1:47134
I20260809 01:46:39.017647 15185 tablet_bootstrap.cc:492] T 1c76ea235a1a4621b2bb4962c6f94656 P 36abc3bb1604427cb977fa2e6a6f4272: No bootstrap required, opened a new log
I20260809 01:46:39.018354 15185 ts_tablet_manager.cc:1403] T 1c76ea235a1a4621b2bb4962c6f94656 P 36abc3bb1604427cb977fa2e6a6f4272: Time spent bootstrapping tablet: real 0.095s user 0.013s sys 0.028s
I20260809 01:46:39.022135 14718 tablet_service.cc:1558] Processing DeleteTablet for tablet e8174401df624fc8bb9fcd4e675042db with delete_type TABLET_DATA_DELETED (replaced by tablet be44f2b9e9fa4ed9812b51e4aafddedf) from {username='slave'} at 127.0.0.1:42244
I20260809 01:46:39.024262 14497 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c76ea235a1a4621b2bb4962c6f94656 with delete_type TABLET_DATA_DELETED (replaced by tablet 27a27ed554414d60a026addeee969bd4) from {username='slave'} at 127.0.0.1:47134
I20260809 01:46:39.027326 14095 catalog_manager.cc:6807] ReplaceTablet: tablet 1c76ea235a1a4621b2bb4962c6f94656 deleted and replaced by tablet 27a27ed554414d60a026addeee969bd4
I20260809 01:46:39.027232 14969 tablet_service.cc:1558] Processing DeleteTablet for tablet e8174401df624fc8bb9fcd4e675042db with delete_type TABLET_DATA_DELETED (replaced by tablet be44f2b9e9fa4ed9812b51e4aafddedf) from {username='slave'} at 127.0.0.1:45636
I20260809 01:46:39.028455 14094 catalog_manager.cc:6807] ReplaceTablet: tablet e8174401df624fc8bb9fcd4e675042db deleted and replaced by tablet be44f2b9e9fa4ed9812b51e4aafddedf
W20260809 01:46:39.030733 14083 catalog_manager.cc:5030] TS 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383): delete failed for tablet e8174401df624fc8bb9fcd4e675042db because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e8174401df624fc8bb9fcd4e675042db already in progress: creating tablet
W20260809 01:46:39.038360 14085 catalog_manager.cc:5030] TS 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981): delete failed for tablet e8174401df624fc8bb9fcd4e675042db because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e8174401df624fc8bb9fcd4e675042db already in progress: creating tablet
W20260809 01:46:39.051276 14082 catalog_manager.cc:5030] TS e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177): delete failed for tablet 1c76ea235a1a4621b2bb4962c6f94656 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c76ea235a1a4621b2bb4962c6f94656 already in progress: creating tablet
I20260809 01:46:39.054605 14496 tablet_service.cc:1511] Processing CreateTablet for tablet 27a27ed554414d60a026addeee969bd4 (DEFAULT_TABLE table=default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 [id=9ec275401ed84e09b74eb2b454e37c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:39.057180 14496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27a27ed554414d60a026addeee969bd4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:46:39.062640 14085 catalog_manager.cc:5030] TS 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981): delete failed for tablet 1c76ea235a1a4621b2bb4962c6f94656 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c76ea235a1a4621b2bb4962c6f94656 already in progress: creating tablet
I20260809 01:46:39.063941 14718 tablet_service.cc:1511] Processing CreateTablet for tablet 27a27ed554414d60a026addeee969bd4 (DEFAULT_TABLE table=default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 [id=9ec275401ed84e09b74eb2b454e37c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:39.065186 14718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27a27ed554414d60a026addeee969bd4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:46:39.073920 14082 catalog_manager.cc:5030] TS e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177): delete failed for tablet e8174401df624fc8bb9fcd4e675042db because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e8174401df624fc8bb9fcd4e675042db already in progress: creating tablet
I20260809 01:46:39.077172 14497 tablet_service.cc:1511] Processing CreateTablet for tablet be44f2b9e9fa4ed9812b51e4aafddedf (DEFAULT_TABLE table=default.table_645ec611a69b474199c9493389824d89 [id=5ed9016f7a8843bdb0dab988c9968d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:39.078279 14497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be44f2b9e9fa4ed9812b51e4aafddedf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:39.086752 14968 tablet_service.cc:1511] Processing CreateTablet for tablet 27a27ed554414d60a026addeee969bd4 (DEFAULT_TABLE table=default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 [id=9ec275401ed84e09b74eb2b454e37c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:39.087864 14968 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27a27ed554414d60a026addeee969bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:39.102193 14717 tablet_service.cc:1511] Processing CreateTablet for tablet be44f2b9e9fa4ed9812b51e4aafddedf (DEFAULT_TABLE table=default.table_645ec611a69b474199c9493389824d89 [id=5ed9016f7a8843bdb0dab988c9968d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:39.103403 14717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be44f2b9e9fa4ed9812b51e4aafddedf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:39.111433 14969 tablet_service.cc:1511] Processing CreateTablet for tablet be44f2b9e9fa4ed9812b51e4aafddedf (DEFAULT_TABLE table=default.table_645ec611a69b474199c9493389824d89 [id=5ed9016f7a8843bdb0dab988c9968d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:39.112607 14969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be44f2b9e9fa4ed9812b51e4aafddedf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:39.124563 15181 tablet_bootstrap.cc:492] T e8174401df624fc8bb9fcd4e675042db P e3d5dac2381748eabc723a0c471ad03f: No bootstrap required, opened a new log
I20260809 01:46:39.124145 15184 raft_consensus.cc:359] T e8174401df624fc8bb9fcd4e675042db P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.125202 15181 ts_tablet_manager.cc:1403] T e8174401df624fc8bb9fcd4e675042db P e3d5dac2381748eabc723a0c471ad03f: Time spent bootstrapping tablet: real 0.206s user 0.019s sys 0.030s
I20260809 01:46:39.125339 15184 raft_consensus.cc:385] T e8174401df624fc8bb9fcd4e675042db P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.125797 15184 raft_consensus.cc:740] T e8174401df624fc8bb9fcd4e675042db P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9336b0d6675b4225831595357c4bdc3b, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.126952 15184 consensus_queue.cc:260] T e8174401df624fc8bb9fcd4e675042db P 9336b0d6675b4225831595357c4bdc3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.136523 14103 catalog_manager.cc:2200] created HMS catalog entry for table [id=f71d960253a1446eae22da13257be951]
I20260809 01:46:39.137068 15185 raft_consensus.cc:359] T 1c76ea235a1a4621b2bb4962c6f94656 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } }
I20260809 01:46:39.140156 15185 raft_consensus.cc:385] T 1c76ea235a1a4621b2bb4962c6f94656 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
F20260809 01:46:39.146783 14305 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 4 received events: NotificationEvent(eventId=1, eventTime=1786239998, eventType=CREATE_TABLE, dbName=default, tableName=table_645ec611a69b474199c9493389824d89, message={"server":"","servicePrincipal":"","db":"default","table":"table_645ec611a69b474199c9493389824d89","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_645ec611a69b474199c9493389824d89\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786239998},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/warehouse/table_645ec611a69b474199c9493389824d89\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_645ec611a69b474199c9493389824d89\",\"kudu.table_id\":\"5ed9016f7a8843bdb0dab988c9968d68\",\"transient_lastDdlTime\":\"1786239998\",\"comment\":\"\",\"kudu.master_addresses\":\"127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"a317462afe1741b5bf318d3556a05e8e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786239998,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1786239998, eventType=CREATE_TABLE, dbName=default, tableName=table_409febef55ad4ddb8638e01e247c19fb, message={"server":"","servicePrincipal":"","db":"default","table":"table_409febef55ad4ddb8638e01e247c19fb","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_409febef55ad4ddb8638e01e247c19fb\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786239998},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/warehouse/table_409febef55ad4ddb8638e01e247c19fb\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_409febef55ad4ddb8638e01e247c19fb\",\"kudu.table_id\":\"762f864272234bad897b360c54af0519\",\"transient_lastDdlTime\":\"1786239998\",\"comment\":\"\",\"kudu.master_addresses\":\"127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"a317462afe1741b5bf318d3556a05e8e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786239998,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260809 01:46:39.155361 15185 raft_consensus.cc:740] T 1c76ea235a1a4621b2bb4962c6f94656 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36abc3bb1604427cb977fa2e6a6f4272, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.156706 15185 consensus_queue.cc:260] T 1c76ea235a1a4621b2bb4962c6f94656 P 36abc3bb1604427cb977fa2e6a6f4272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } }
I20260809 01:46:39.174204 14103 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33272:
table { table_name: "default.Table_ed71d957cdb34b9fa97483de90b1d539" } modify_external_catalogs: true
I20260809 01:46:39.177601 14103 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33272:
table { table_name: "default.Table_ed71d957cdb34b9fa97483de90b1d539" } modify_external_catalogs: true
I20260809 01:46:39.188927 15185 ts_tablet_manager.cc:1434] T 1c76ea235a1a4621b2bb4962c6f94656 P 36abc3bb1604427cb977fa2e6a6f4272: Time spent starting tablet: real 0.170s user 0.038s sys 0.026s
I20260809 01:46:39.195221 15184 ts_tablet_manager.cc:1434] T e8174401df624fc8bb9fcd4e675042db P 9336b0d6675b4225831595357c4bdc3b: Time spent starting tablet: real 0.174s user 0.032s sys 0.005s
I20260809 01:46:39.195175 14591 heartbeater.cc:499] Master 127.10.8.125:43535 was elected leader, sending a full tablet report...
I20260809 01:46:39.196274 15184 tablet_bootstrap.cc:492] T 1c76ea235a1a4621b2bb4962c6f94656 P 9336b0d6675b4225831595357c4bdc3b: Bootstrap starting.
W20260809 01:46:39.202342 14094 catalog_manager.cc:5552] Ignoring report from unknown tablet 1c76ea235a1a4621b2bb4962c6f94656
I20260809 01:46:39.202822 15184 tablet_bootstrap.cc:654] T 1c76ea235a1a4621b2bb4962c6f94656 P 9336b0d6675b4225831595357c4bdc3b: Neither blocks nor log segments found. Creating new log.
W20260809 01:46:39.202968 14094 catalog_manager.cc:5552] Ignoring report from unknown tablet e8174401df624fc8bb9fcd4e675042db
I20260809 01:46:39.203008 15185 tablet_bootstrap.cc:492] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272: Bootstrap starting.
W20260809 01:46:39.204062 14095 catalog_manager.cc:5552] Ignoring report from unknown tablet 1c76ea235a1a4621b2bb4962c6f94656
I20260809 01:46:39.209100 15185 tablet_bootstrap.cc:654] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272: Neither blocks nor log segments found. Creating new log.
W20260809 01:46:39.210651 14095 catalog_manager.cc:5552] Ignoring report from unknown tablet 1c76ea235a1a4621b2bb4962c6f94656
I20260809 01:46:39.224819 15185 tablet_bootstrap.cc:492] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272: No bootstrap required, opened a new log
I20260809 01:46:39.225406 15185 ts_tablet_manager.cc:1403] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272: Time spent bootstrapping tablet: real 0.023s user 0.008s sys 0.008s
I20260809 01:46:39.227555 15184 tablet_bootstrap.cc:492] T 1c76ea235a1a4621b2bb4962c6f94656 P 9336b0d6675b4225831595357c4bdc3b: No bootstrap required, opened a new log
I20260809 01:46:39.228004 15184 ts_tablet_manager.cc:1403] T 1c76ea235a1a4621b2bb4962c6f94656 P 9336b0d6675b4225831595357c4bdc3b: Time spent bootstrapping tablet: real 0.032s user 0.007s sys 0.006s
I20260809 01:46:39.228638 15185 raft_consensus.cc:359] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.235589 14718 tablet_service.cc:1511] Processing CreateTablet for tablet 2e7e3e285bc64edb9b47e11123f2ba63 (DEFAULT_TABLE table=default.table_ed71d957cdb34b9fa97483de90b1d539 [id=f71d960253a1446eae22da13257be951]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:39.236680 14718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e7e3e285bc64edb9b47e11123f2ba63. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f8102d1f920 google::LogMessage::Flush() at ??:0
I20260809 01:46:39.238260 14497 tablet_service.cc:1511] Processing CreateTablet for tablet 2e7e3e285bc64edb9b47e11123f2ba63 (DEFAULT_TABLE table=default.table_ed71d957cdb34b9fa97483de90b1d539 [id=f71d960253a1446eae22da13257be951]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:39.239413 14497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e7e3e285bc64edb9b47e11123f2ba63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:39.243028 14095 master_service.cc:960] ReplaceTablet: received request to replace tablet 1c76ea235a1a4621b2bb4962c6f94656 from {username='slave'} at 127.0.0.1:33272
I20260809 01:46:39.243238 14968 tablet_service.cc:1511] Processing CreateTablet for tablet 2e7e3e285bc64edb9b47e11123f2ba63 (DEFAULT_TABLE table=default.table_ed71d957cdb34b9fa97483de90b1d539 [id=f71d960253a1446eae22da13257be951]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:39.244359 14968 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e7e3e285bc64edb9b47e11123f2ba63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:39.249846 15184 raft_consensus.cc:359] T 1c76ea235a1a4621b2bb4962c6f94656 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } }
I20260809 01:46:39.250520 15184 raft_consensus.cc:385] T 1c76ea235a1a4621b2bb4962c6f94656 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.250780 15184 raft_consensus.cc:740] T 1c76ea235a1a4621b2bb4962c6f94656 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9336b0d6675b4225831595357c4bdc3b, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.251578 15184 consensus_queue.cc:260] T 1c76ea235a1a4621b2bb4962c6f94656 P 9336b0d6675b4225831595357c4bdc3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } }
W20260809 01:46:39.256162 14094 catalog_manager.cc:5552] Ignoring report from unknown tablet 1c76ea235a1a4621b2bb4962c6f94656
I20260809 01:46:39.261152 15184 ts_tablet_manager.cc:1434] T 1c76ea235a1a4621b2bb4962c6f94656 P 9336b0d6675b4225831595357c4bdc3b: Time spent starting tablet: real 0.033s user 0.007s sys 0.000s
I20260809 01:46:39.266198 14094 master_service.cc:960] ReplaceTablet: received request to replace tablet 27a27ed554414d60a026addeee969bd4 from {username='slave'} at 127.0.0.1:33272
W20260809 01:46:39.266456 14095 catalog_manager.cc:5552] Ignoring report from unknown tablet 1c76ea235a1a4621b2bb4962c6f94656
I20260809 01:46:39.269510 15185 raft_consensus.cc:385] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.269925 15185 raft_consensus.cc:740] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36abc3bb1604427cb977fa2e6a6f4272, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.271565 15185 consensus_queue.cc:260] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.277895 15184 tablet_bootstrap.cc:492] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b: Bootstrap starting.
W20260809 01:46:39.283937 14095 catalog_manager.cc:5552] Ignoring report from unknown tablet e8174401df624fc8bb9fcd4e675042db
I20260809 01:46:39.283959 15185 ts_tablet_manager.cc:1434] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272: Time spent starting tablet: real 0.058s user 0.005s sys 0.000s
I20260809 01:46:39.284819 15185 tablet_bootstrap.cc:492] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272: Bootstrap starting.
W20260809 01:46:39.288582 14095 catalog_manager.cc:5552] Ignoring report from unknown tablet e8174401df624fc8bb9fcd4e675042db
@ 0x7f8102d1f68f google::LogMessage::~LogMessage() at ??:0
I20260809 01:46:39.296801 15184 tablet_bootstrap.cc:654] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:39.298668 15185 tablet_bootstrap.cc:654] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:39.307276 14094 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 3d2c2e2b6aa54060a87795a74d36e4d4: Sending DeleteTablet for 3 replicas of tablet 27a27ed554414d60a026addeee969bd4
W20260809 01:46:39.308348 14601 tablet.cc:2406] T 1c76ea235a1a4621b2bb4962c6f94656 P 9336b0d6675b4225831595357c4bdc3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:46:39.308965 14094 catalog_manager.cc:6807] ReplaceTablet: tablet 27a27ed554414d60a026addeee969bd4 deleted and replaced by tablet af393a465cb34dbdb4795df8ae8c49a4
I20260809 01:46:39.311504 14717 tablet_service.cc:1558] Processing DeleteTablet for tablet 27a27ed554414d60a026addeee969bd4 with delete_type TABLET_DATA_DELETED (replaced by tablet af393a465cb34dbdb4795df8ae8c49a4) from {username='slave'} at 127.0.0.1:42244
I20260809 01:46:39.312383 14496 tablet_service.cc:1558] Processing DeleteTablet for tablet 27a27ed554414d60a026addeee969bd4 with delete_type TABLET_DATA_DELETED (replaced by tablet af393a465cb34dbdb4795df8ae8c49a4) from {username='slave'} at 127.0.0.1:47134
W20260809 01:46:39.315160 14085 catalog_manager.cc:5030] TS 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981): delete failed for tablet 27a27ed554414d60a026addeee969bd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27a27ed554414d60a026addeee969bd4 already in progress: creating tablet
I20260809 01:46:39.316860 14969 tablet_service.cc:1558] Processing DeleteTablet for tablet 27a27ed554414d60a026addeee969bd4 with delete_type TABLET_DATA_DELETED (replaced by tablet af393a465cb34dbdb4795df8ae8c49a4) from {username='slave'} at 127.0.0.1:45636
W20260809 01:46:39.315359 14083 catalog_manager.cc:5030] TS 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383): delete failed for tablet 27a27ed554414d60a026addeee969bd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27a27ed554414d60a026addeee969bd4 already in progress: creating tablet
W20260809 01:46:39.319417 14082 catalog_manager.cc:5030] TS e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177): delete failed for tablet 27a27ed554414d60a026addeee969bd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27a27ed554414d60a026addeee969bd4 already in progress: creating tablet
I20260809 01:46:39.325436 15181 raft_consensus.cc:359] T e8174401df624fc8bb9fcd4e675042db P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.326439 15181 raft_consensus.cc:385] T e8174401df624fc8bb9fcd4e675042db P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.326803 15181 raft_consensus.cc:740] T e8174401df624fc8bb9fcd4e675042db P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3d5dac2381748eabc723a0c471ad03f, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.328009 15181 consensus_queue.cc:260] T e8174401df624fc8bb9fcd4e675042db P e3d5dac2381748eabc723a0c471ad03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.344785 14969 tablet_service.cc:1511] Processing CreateTablet for tablet af393a465cb34dbdb4795df8ae8c49a4 (DEFAULT_TABLE table=default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 [id=9ec275401ed84e09b74eb2b454e37c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:39.345880 14969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af393a465cb34dbdb4795df8ae8c49a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:39.349407 14497 tablet_service.cc:1511] Processing CreateTablet for tablet af393a465cb34dbdb4795df8ae8c49a4 (DEFAULT_TABLE table=default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 [id=9ec275401ed84e09b74eb2b454e37c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:39.350797 14718 tablet_service.cc:1511] Processing CreateTablet for tablet af393a465cb34dbdb4795df8ae8c49a4 (DEFAULT_TABLE table=default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 [id=9ec275401ed84e09b74eb2b454e37c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:39.351858 14718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af393a465cb34dbdb4795df8ae8c49a4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 01:46:39.352320 14094 catalog_manager.cc:5552] Ignoring report from unknown tablet e8174401df624fc8bb9fcd4e675042db
I20260809 01:46:39.357262 14497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af393a465cb34dbdb4795df8ae8c49a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:39.361016 15185 tablet_bootstrap.cc:492] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272: No bootstrap required, opened a new log
I20260809 01:46:39.361466 15185 ts_tablet_manager.cc:1403] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272: Time spent bootstrapping tablet: real 0.077s user 0.011s sys 0.003s
I20260809 01:46:39.361865 15181 ts_tablet_manager.cc:1434] T e8174401df624fc8bb9fcd4e675042db P e3d5dac2381748eabc723a0c471ad03f: Time spent starting tablet: real 0.236s user 0.037s sys 0.012s
I20260809 01:46:39.364521 15184 tablet_bootstrap.cc:492] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b: No bootstrap required, opened a new log
I20260809 01:46:39.364382 15185 raft_consensus.cc:359] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.365037 15184 ts_tablet_manager.cc:1403] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b: Time spent bootstrapping tablet: real 0.087s user 0.008s sys 0.003s
I20260809 01:46:39.365057 15185 raft_consensus.cc:385] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 01:46:39.365540 14094 catalog_manager.cc:5552] Ignoring report from unknown tablet e8174401df624fc8bb9fcd4e675042db
I20260809 01:46:39.367915 15184 raft_consensus.cc:359] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.372715 15184 raft_consensus.cc:385] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.373044 15184 raft_consensus.cc:740] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9336b0d6675b4225831595357c4bdc3b, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.365350 15185 raft_consensus.cc:740] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36abc3bb1604427cb977fa2e6a6f4272, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.375494 14094 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33272:
name: "default.Table_042c570880b14f5f9317fe65d87c7cf5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:39.375855 15185 consensus_queue.cc:260] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.377143 15181 tablet_bootstrap.cc:492] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f: Bootstrap starting.
I20260809 01:46:39.376696 15184 consensus_queue.cc:260] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.385628 14496 tablet_service.cc:1558] Processing DeleteTablet for tablet 27a27ed554414d60a026addeee969bd4 with delete_type TABLET_DATA_DELETED (replaced by tablet af393a465cb34dbdb4795df8ae8c49a4) from {username='slave'} at 127.0.0.1:47134
I20260809 01:46:39.386658 15181 tablet_bootstrap.cc:654] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f: Neither blocks nor log segments found. Creating new log.
W20260809 01:46:39.389657 14844 tablet.cc:2406] T 1c76ea235a1a4621b2bb4962c6f94656 P 36abc3bb1604427cb977fa2e6a6f4272: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 01:46:39.390229 14085 catalog_manager.cc:5030] TS 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981): delete failed for tablet 27a27ed554414d60a026addeee969bd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27a27ed554414d60a026addeee969bd4 already in progress: creating tablet
I20260809 01:46:39.393100 15185 ts_tablet_manager.cc:1434] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272: Time spent starting tablet: real 0.031s user 0.004s sys 0.002s
I20260809 01:46:39.406106 15185 tablet_bootstrap.cc:492] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272: Bootstrap starting.
I20260809 01:46:39.407171 15184 ts_tablet_manager.cc:1434] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b: Time spent starting tablet: real 0.042s user 0.006s sys 0.000s
I20260809 01:46:39.415330 14496 tablet_service.cc:1558] Processing DeleteTablet for tablet 27a27ed554414d60a026addeee969bd4 with delete_type TABLET_DATA_DELETED (replaced by tablet af393a465cb34dbdb4795df8ae8c49a4) from {username='slave'} at 127.0.0.1:47134
I20260809 01:46:39.416160 15191 tablet_replica.cc:333] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b: stopping tablet replica
I20260809 01:46:39.417022 15191 raft_consensus.cc:2243] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Raft consensus shutting down.
@ 0x7f810de9ffbc kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260809 01:46:39.417546 15191 raft_consensus.cc:2272] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:39.420734 15191 ts_tablet_manager.cc:1916] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:39.436558 15185 tablet_bootstrap.cc:654] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272: Neither blocks nor log segments found. Creating new log.
W20260809 01:46:39.444923 15154 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:46:39.445694 15154 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:46:39.446455 15154 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 01:46:39.447254 15184 tablet_bootstrap.cc:492] T be44f2b9e9fa4ed9812b51e4aafddedf P 9336b0d6675b4225831595357c4bdc3b: Bootstrap starting.
I20260809 01:46:39.454247 15181 tablet_bootstrap.cc:492] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f: No bootstrap required, opened a new log
I20260809 01:46:39.454689 14090 master_service.cc:960] ReplaceTablet: received request to replace tablet e8174401df624fc8bb9fcd4e675042db from {username='slave'} at 127.0.0.1:33272
I20260809 01:46:39.455657 15181 ts_tablet_manager.cc:1403] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f: Time spent bootstrapping tablet: real 0.079s user 0.020s sys 0.033s
I20260809 01:46:39.456725 15184 tablet_bootstrap.cc:654] T be44f2b9e9fa4ed9812b51e4aafddedf P 9336b0d6675b4225831595357c4bdc3b: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:39.458274 15181 raft_consensus.cc:359] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } }
I20260809 01:46:39.458938 15181 raft_consensus.cc:385] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.459229 15181 raft_consensus.cc:740] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3d5dac2381748eabc723a0c471ad03f, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.460005 15191 ts_tablet_manager.cc:1929] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:46:39.461447 15185 tablet_bootstrap.cc:492] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272: No bootstrap required, opened a new log
I20260809 01:46:39.461933 15185 ts_tablet_manager.cc:1403] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272: Time spent bootstrapping tablet: real 0.056s user 0.007s sys 0.007s
I20260809 01:46:39.463066 15191 log.cc:1199] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/27a27ed554414d60a026addeee969bd4
I20260809 01:46:39.464681 15191 ts_tablet_manager.cc:1950] T 27a27ed554414d60a026addeee969bd4 P 9336b0d6675b4225831595357c4bdc3b: Deleting consensus metadata
I20260809 01:46:39.464788 15185 raft_consensus.cc:359] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.459975 15181 consensus_queue.cc:260] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } }
I20260809 01:46:39.465466 15185 raft_consensus.cc:385] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.466018 15185 raft_consensus.cc:740] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36abc3bb1604427cb977fa2e6a6f4272, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.466838 15185 consensus_queue.cc:260] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.467379 15181 ts_tablet_manager.cc:1434] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f: Time spent starting tablet: real 0.011s user 0.004s sys 0.002s
I20260809 01:46:39.468197 15181 tablet_bootstrap.cc:492] T be44f2b9e9fa4ed9812b51e4aafddedf P e3d5dac2381748eabc723a0c471ad03f: Bootstrap starting.
I20260809 01:46:39.468384 14085 catalog_manager.cc:5050] TS 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981): tablet 27a27ed554414d60a026addeee969bd4 (table default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 [id=9ec275401ed84e09b74eb2b454e37c4f]) successfully deleted
I20260809 01:46:39.472576 15185 ts_tablet_manager.cc:1434] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260809 01:46:39.473475 15185 tablet_bootstrap.cc:492] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272: Bootstrap starting.
I20260809 01:46:39.477558 14718 tablet_service.cc:1558] Processing DeleteTablet for tablet 27a27ed554414d60a026addeee969bd4 with delete_type TABLET_DATA_DELETED (replaced by tablet af393a465cb34dbdb4795df8ae8c49a4) from {username='slave'} at 127.0.0.1:42244
W20260809 01:46:39.477653 15088 tablet.cc:2406] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:46:39.478528 15189 tablet_replica.cc:333] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272: stopping tablet replica
I20260809 01:46:39.479452 15189 raft_consensus.cc:2243] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:39.480011 15189 raft_consensus.cc:2272] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:39.483662 15189 ts_tablet_manager.cc:1916] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 01:46:39.491289 14095 catalog_manager.cc:5552] Ignoring report from unknown tablet 1c76ea235a1a4621b2bb4962c6f94656
I20260809 01:46:39.494097 15185 tablet_bootstrap.cc:654] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:39.495138 15181 tablet_bootstrap.cc:654] T be44f2b9e9fa4ed9812b51e4aafddedf P e3d5dac2381748eabc723a0c471ad03f: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:39.500113 15184 tablet_bootstrap.cc:492] T be44f2b9e9fa4ed9812b51e4aafddedf P 9336b0d6675b4225831595357c4bdc3b: No bootstrap required, opened a new log
I20260809 01:46:39.500661 15184 ts_tablet_manager.cc:1403] T be44f2b9e9fa4ed9812b51e4aafddedf P 9336b0d6675b4225831595357c4bdc3b: Time spent bootstrapping tablet: real 0.054s user 0.010s sys 0.001s
W20260809 01:46:39.503309 14090 catalog_manager.cc:5552] Ignoring report from unknown tablet 1c76ea235a1a4621b2bb4962c6f94656
@ 0x7f810de9f77c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260809 01:46:39.509766 15184 raft_consensus.cc:359] T be44f2b9e9fa4ed9812b51e4aafddedf P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.510520 15184 raft_consensus.cc:385] T be44f2b9e9fa4ed9812b51e4aafddedf P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.510821 15184 raft_consensus.cc:740] T be44f2b9e9fa4ed9812b51e4aafddedf P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9336b0d6675b4225831595357c4bdc3b, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.511555 15184 consensus_queue.cc:260] T be44f2b9e9fa4ed9812b51e4aafddedf P 9336b0d6675b4225831595357c4bdc3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.528834 14090 master_service.cc:960] ReplaceTablet: received request to replace tablet e8174401df624fc8bb9fcd4e675042db from {username='slave'} at 127.0.0.1:33272
I20260809 01:46:39.529325 15181 tablet_bootstrap.cc:492] T be44f2b9e9fa4ed9812b51e4aafddedf P e3d5dac2381748eabc723a0c471ad03f: No bootstrap required, opened a new log
I20260809 01:46:39.529752 15181 ts_tablet_manager.cc:1403] T be44f2b9e9fa4ed9812b51e4aafddedf P e3d5dac2381748eabc723a0c471ad03f: Time spent bootstrapping tablet: real 0.062s user 0.008s sys 0.003s
I20260809 01:46:39.532174 15181 raft_consensus.cc:359] T be44f2b9e9fa4ed9812b51e4aafddedf P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.517607 15184 ts_tablet_manager.cc:1434] T be44f2b9e9fa4ed9812b51e4aafddedf P 9336b0d6675b4225831595357c4bdc3b: Time spent starting tablet: real 0.017s user 0.006s sys 0.001s
I20260809 01:46:39.532828 15181 raft_consensus.cc:385] T be44f2b9e9fa4ed9812b51e4aafddedf P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.533073 15181 raft_consensus.cc:740] T be44f2b9e9fa4ed9812b51e4aafddedf P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3d5dac2381748eabc723a0c471ad03f, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.533775 15184 tablet_bootstrap.cc:492] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b: Bootstrap starting.
I20260809 01:46:39.533887 15181 consensus_queue.cc:260] T be44f2b9e9fa4ed9812b51e4aafddedf P e3d5dac2381748eabc723a0c471ad03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.536337 15181 ts_tablet_manager.cc:1434] T be44f2b9e9fa4ed9812b51e4aafddedf P e3d5dac2381748eabc723a0c471ad03f: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260809 01:46:39.538904 15189 ts_tablet_manager.cc:1929] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:46:39.539346 15189 log.cc:1199] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/27a27ed554414d60a026addeee969bd4
I20260809 01:46:39.540263 15189 ts_tablet_manager.cc:1950] T 27a27ed554414d60a026addeee969bd4 P 36abc3bb1604427cb977fa2e6a6f4272: Deleting consensus metadata
@ 0x7f810dea37f2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260809 01:46:39.559821 15181 tablet_bootstrap.cc:492] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f: Bootstrap starting.
W20260809 01:46:39.565419 15154 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260809 01:46:39.565743 15154 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260809 01:46:39.566076 15154 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:46:39.566459 15154 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:46:39.566773 15154 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:46:39.567073 15154 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:46:39.567766 14083 catalog_manager.cc:5050] TS 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383): tablet 27a27ed554414d60a026addeee969bd4 (table default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 [id=9ec275401ed84e09b74eb2b454e37c4f]) successfully deleted
I20260809 01:46:39.567975 15181 tablet_bootstrap.cc:654] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f: Neither blocks nor log segments found. Creating new log.
@ 0x7f810dea37aa _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260809 01:46:39.585541 15181 tablet_bootstrap.cc:492] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f: No bootstrap required, opened a new log
I20260809 01:46:39.585927 15181 ts_tablet_manager.cc:1403] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f: Time spent bootstrapping tablet: real 0.026s user 0.012s sys 0.000s
I20260809 01:46:39.591300 15181 raft_consensus.cc:359] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.591881 15181 raft_consensus.cc:385] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.592115 15181 raft_consensus.cc:740] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3d5dac2381748eabc723a0c471ad03f, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.597755 15181 consensus_queue.cc:260] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.605738 15181 ts_tablet_manager.cc:1434] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f: Time spent starting tablet: real 0.020s user 0.003s sys 0.003s
I20260809 01:46:39.606700 15181 tablet_bootstrap.cc:492] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f: Bootstrap starting.
@ 0x7f810dea373a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260809 01:46:39.611639 14969 tablet_service.cc:1558] Processing DeleteTablet for tablet 27a27ed554414d60a026addeee969bd4 with delete_type TABLET_DATA_DELETED (replaced by tablet af393a465cb34dbdb4795df8ae8c49a4) from {username='slave'} at 127.0.0.1:45636
I20260809 01:46:39.612093 15181 tablet_bootstrap.cc:654] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:39.612527 15190 tablet_replica.cc:333] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f: stopping tablet replica
I20260809 01:46:39.613551 15190 raft_consensus.cc:2243] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:39.614111 15190 raft_consensus.cc:2272] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:39.617614 15190 ts_tablet_manager.cc:1916] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:39.626196 15185 tablet_bootstrap.cc:492] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272: No bootstrap required, opened a new log
I20260809 01:46:39.626729 15185 ts_tablet_manager.cc:1403] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272: Time spent bootstrapping tablet: real 0.153s user 0.008s sys 0.004s
I20260809 01:46:39.631259 15184 tablet_bootstrap.cc:654] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:39.631701 15185 raft_consensus.cc:359] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
@ 0x7f810dea3702 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260809 01:46:39.633638 15185 raft_consensus.cc:385] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.633919 15185 raft_consensus.cc:740] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36abc3bb1604427cb977fa2e6a6f4272, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.640527 15185 consensus_queue.cc:260] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.645670 15190 ts_tablet_manager.cc:1929] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 01:46:39.646190 15190 log.cc:1199] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/27a27ed554414d60a026addeee969bd4
I20260809 01:46:39.647157 15190 ts_tablet_manager.cc:1950] T 27a27ed554414d60a026addeee969bd4 P e3d5dac2381748eabc723a0c471ad03f: Deleting consensus metadata
I20260809 01:46:39.653355 15185 ts_tablet_manager.cc:1434] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272: Time spent starting tablet: real 0.026s user 0.005s sys 0.001s
I20260809 01:46:39.658897 15181 tablet_bootstrap.cc:492] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f: No bootstrap required, opened a new log
I20260809 01:46:39.659309 15181 ts_tablet_manager.cc:1403] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f: Time spent bootstrapping tablet: real 0.053s user 0.010s sys 0.001s
I20260809 01:46:39.663477 14090 master_service.cc:960] ReplaceTablet: received request to replace tablet 1c76ea235a1a4621b2bb4962c6f94656 from {username='slave'} at 127.0.0.1:33272
I20260809 01:46:39.666182 14082 catalog_manager.cc:5050] TS e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177): tablet 27a27ed554414d60a026addeee969bd4 (table default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 [id=9ec275401ed84e09b74eb2b454e37c4f]) successfully deleted
I20260809 01:46:39.662055 15181 raft_consensus.cc:359] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.667348 15181 raft_consensus.cc:385] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.667651 15181 raft_consensus.cc:740] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3d5dac2381748eabc723a0c471ad03f, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.668354 15181 consensus_queue.cc:260] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.672029 15185 tablet_bootstrap.cc:492] T af393a465cb34dbdb4795df8ae8c49a4 P 36abc3bb1604427cb977fa2e6a6f4272: Bootstrap starting.
@ 0x7f810dea29fe std::__1::__function::__func<>::operator()() at ??:0
@ 0x5bd8c5 std::__1::__function::__value_func<>::operator()() at ??:?
I20260809 01:46:39.681993 15181 ts_tablet_manager.cc:1434] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f: Time spent starting tablet: real 0.022s user 0.007s sys 0.001s
@ 0x5bd6fa std::__1::function<>::operator()() at ??:?
I20260809 01:46:39.683640 15184 tablet_bootstrap.cc:492] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b: No bootstrap required, opened a new log
I20260809 01:46:39.684058 15184 ts_tablet_manager.cc:1403] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b: Time spent bootstrapping tablet: real 0.151s user 0.009s sys 0.002s
I20260809 01:46:39.686934 15184 raft_consensus.cc:359] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.687656 15184 raft_consensus.cc:385] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.687937 15184 raft_consensus.cc:740] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9336b0d6675b4225831595357c4bdc3b, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.688655 15184 consensus_queue.cc:260] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.690452 15154 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.8.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:35729
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.124:34129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.8.124
--webserver_port=44759
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 01:43:22 UTC on e7f111948823
build id 13251
TSAN enabled
I20260809 01:46:39.692996 15181 tablet_bootstrap.cc:492] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f: Bootstrap starting.
I20260809 01:46:39.699448 15181 tablet_bootstrap.cc:654] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:39.692189 15154 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:46:39.703810 15154 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 01:46:39.707739 15184 ts_tablet_manager.cc:1434] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20260809 01:46:39.708721 15184 tablet_bootstrap.cc:492] T af393a465cb34dbdb4795df8ae8c49a4 P 9336b0d6675b4225831595357c4bdc3b: Bootstrap starting.
I20260809 01:46:39.709064 15209 raft_consensus.cc:493] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:39.709602 15209 raft_consensus.cc:515] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.712520 15209 leader_election.cc:290] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981), 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383)
I20260809 01:46:39.713748 15185 tablet_bootstrap.cc:654] T af393a465cb34dbdb4795df8ae8c49a4 P 36abc3bb1604427cb977fa2e6a6f4272: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:39.714056 15193 raft_consensus.cc:493] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:39.714560 15193 raft_consensus.cc:515] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.719563 15193 leader_election.cc:290] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981), e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177)
I20260809 01:46:39.731067 15192 raft_consensus.cc:493] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:39.731607 15192 raft_consensus.cc:515] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.735076 15192 leader_election.cc:290] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383), e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177)
W20260809 01:46:39.739151 14081 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.124:34129, user_credentials={real_user=slave}} blocked reactor thread for 44593.4us
I20260809 01:46:39.744632 15185 tablet_bootstrap.cc:492] T af393a465cb34dbdb4795df8ae8c49a4 P 36abc3bb1604427cb977fa2e6a6f4272: No bootstrap required, opened a new log
I20260809 01:46:39.745160 15185 ts_tablet_manager.cc:1403] T af393a465cb34dbdb4795df8ae8c49a4 P 36abc3bb1604427cb977fa2e6a6f4272: Time spent bootstrapping tablet: real 0.076s user 0.006s sys 0.004s
@ 0x7f8104f091eb kudu::Thread::SuperviseThread() at ??:0
@ 0x4568e0 __tsan_thread_start_func at ??:?
I20260809 01:46:39.749184 15185 raft_consensus.cc:359] T af393a465cb34dbdb4795df8ae8c49a4 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } }
I20260809 01:46:39.749907 15185 raft_consensus.cc:385] T af393a465cb34dbdb4795df8ae8c49a4 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.750273 15185 raft_consensus.cc:740] T af393a465cb34dbdb4795df8ae8c49a4 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36abc3bb1604427cb977fa2e6a6f4272, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.750964 15185 consensus_queue.cc:260] T af393a465cb34dbdb4795df8ae8c49a4 P 36abc3bb1604427cb977fa2e6a6f4272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } }
I20260809 01:46:39.755017 15184 tablet_bootstrap.cc:654] T af393a465cb34dbdb4795df8ae8c49a4 P 9336b0d6675b4225831595357c4bdc3b: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:39.772118 14090 master_service.cc:960] ReplaceTablet: received request to replace tablet 1c76ea235a1a4621b2bb4962c6f94656 from {username='slave'} at 127.0.0.1:33272
I20260809 01:46:39.780696 15185 ts_tablet_manager.cc:1434] T af393a465cb34dbdb4795df8ae8c49a4 P 36abc3bb1604427cb977fa2e6a6f4272: Time spent starting tablet: real 0.035s user 0.008s sys 0.000s
I20260809 01:46:39.794334 15209 raft_consensus.cc:493] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:39.794731 15209 raft_consensus.cc:515] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } }
I20260809 01:46:39.796617 15209 leader_election.cc:290] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383), 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981)
I20260809 01:46:39.798290 15181 tablet_bootstrap.cc:492] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f: No bootstrap required, opened a new log
I20260809 01:46:39.798336 15184 tablet_bootstrap.cc:492] T af393a465cb34dbdb4795df8ae8c49a4 P 9336b0d6675b4225831595357c4bdc3b: No bootstrap required, opened a new log
I20260809 01:46:39.798112 14755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e7e3e285bc64edb9b47e11123f2ba63" candidate_uuid: "e3d5dac2381748eabc723a0c471ad03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36abc3bb1604427cb977fa2e6a6f4272" is_pre_election: true
I20260809 01:46:39.798404 14753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c76ea235a1a4621b2bb4962c6f94656" candidate_uuid: "e3d5dac2381748eabc723a0c471ad03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36abc3bb1604427cb977fa2e6a6f4272" is_pre_election: true
I20260809 01:46:39.798702 15181 ts_tablet_manager.cc:1403] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f: Time spent bootstrapping tablet: real 0.106s user 0.017s sys 0.016s
I20260809 01:46:39.798741 15184 ts_tablet_manager.cc:1403] T af393a465cb34dbdb4795df8ae8c49a4 P 9336b0d6675b4225831595357c4bdc3b: Time spent bootstrapping tablet: real 0.090s user 0.013s sys 0.000s
I20260809 01:46:39.798913 14755 raft_consensus.cc:2468] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3d5dac2381748eabc723a0c471ad03f in term 0.
I20260809 01:46:39.799062 14753 raft_consensus.cc:2468] T 1c76ea235a1a4621b2bb4962c6f94656 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3d5dac2381748eabc723a0c471ad03f in term 0.
I20260809 01:46:39.800460 14888 leader_election.cc:304] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36abc3bb1604427cb977fa2e6a6f4272, e3d5dac2381748eabc723a0c471ad03f; no voters:
I20260809 01:46:39.801403 15181 raft_consensus.cc:359] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } }
I20260809 01:46:39.801925 15209 raft_consensus.cc:2804] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:39.802018 15181 raft_consensus.cc:385] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.802279 15209 raft_consensus.cc:493] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:39.802647 15209 raft_consensus.cc:3060] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:39.804123 14888 leader_election.cc:304] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36abc3bb1604427cb977fa2e6a6f4272, e3d5dac2381748eabc723a0c471ad03f; no voters:
I20260809 01:46:39.808161 15193 raft_consensus.cc:493] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:39.802277 15181 raft_consensus.cc:740] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3d5dac2381748eabc723a0c471ad03f, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.808570 15193 raft_consensus.cc:515] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.809719 15181 consensus_queue.cc:260] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } }
I20260809 01:46:39.810495 15193 leader_election.cc:290] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981), e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177)
I20260809 01:46:39.819353 15209 raft_consensus.cc:515] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.813665 14527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8174401df624fc8bb9fcd4e675042db" candidate_uuid: "36abc3bb1604427cb977fa2e6a6f4272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336b0d6675b4225831595357c4bdc3b" is_pre_election: true
I20260809 01:46:39.822300 14527 raft_consensus.cc:2468] T e8174401df624fc8bb9fcd4e675042db P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36abc3bb1604427cb977fa2e6a6f4272 in term 0.
I20260809 01:46:39.814832 14525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be44f2b9e9fa4ed9812b51e4aafddedf" candidate_uuid: "36abc3bb1604427cb977fa2e6a6f4272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336b0d6675b4225831595357c4bdc3b" is_pre_election: true
I20260809 01:46:39.823781 14525 raft_consensus.cc:2468] T be44f2b9e9fa4ed9812b51e4aafddedf P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36abc3bb1604427cb977fa2e6a6f4272 in term 0.
I20260809 01:46:39.824859 15181 ts_tablet_manager.cc:1434] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f: Time spent starting tablet: real 0.026s user 0.007s sys 0.000s
I20260809 01:46:39.815723 15273 raft_consensus.cc:2804] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:39.825922 15273 raft_consensus.cc:493] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:39.826239 15273 raft_consensus.cc:3060] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:39.826285 14648 leader_election.cc:304] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36abc3bb1604427cb977fa2e6a6f4272, 9336b0d6675b4225831595357c4bdc3b; no voters:
I20260809 01:46:39.829967 14522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e7e3e285bc64edb9b47e11123f2ba63" candidate_uuid: "e3d5dac2381748eabc723a0c471ad03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336b0d6675b4225831595357c4bdc3b"
I20260809 01:46:39.830557 14522 raft_consensus.cc:3060] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:39.833843 14525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e7e3e285bc64edb9b47e11123f2ba63" candidate_uuid: "e3d5dac2381748eabc723a0c471ad03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336b0d6675b4225831595357c4bdc3b" is_pre_election: true
I20260809 01:46:39.835712 14527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c76ea235a1a4621b2bb4962c6f94656" candidate_uuid: "e3d5dac2381748eabc723a0c471ad03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336b0d6675b4225831595357c4bdc3b" is_pre_election: true
I20260809 01:46:39.836196 14527 raft_consensus.cc:2468] T 1c76ea235a1a4621b2bb4962c6f94656 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3d5dac2381748eabc723a0c471ad03f in term 0.
I20260809 01:46:39.841748 15193 raft_consensus.cc:2804] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:39.842226 15193 raft_consensus.cc:493] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:39.842654 15193 raft_consensus.cc:3060] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:46:39.849336 15267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:46:39.853940 14648 leader_election.cc:304] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36abc3bb1604427cb977fa2e6a6f4272, 9336b0d6675b4225831595357c4bdc3b; no voters:
I20260809 01:46:39.856671 14522 raft_consensus.cc:2468] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3d5dac2381748eabc723a0c471ad03f in term 1.
I20260809 01:46:39.801496 15184 raft_consensus.cc:359] T af393a465cb34dbdb4795df8ae8c49a4 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } }
I20260809 01:46:39.858336 15281 raft_consensus.cc:2804] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:39.858538 15184 raft_consensus.cc:385] T af393a465cb34dbdb4795df8ae8c49a4 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:39.858713 15281 raft_consensus.cc:493] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:39.858824 15184 raft_consensus.cc:740] T af393a465cb34dbdb4795df8ae8c49a4 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9336b0d6675b4225831595357c4bdc3b, State: Initialized, Role: FOLLOWER
I20260809 01:46:39.859040 15281 raft_consensus.cc:3060] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:39.860936 14889 leader_election.cc:304] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9336b0d6675b4225831595357c4bdc3b, e3d5dac2381748eabc723a0c471ad03f; no voters:
I20260809 01:46:39.862066 15184 consensus_queue.cc:260] T af393a465cb34dbdb4795df8ae8c49a4 P 9336b0d6675b4225831595357c4bdc3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } }
I20260809 01:46:39.864360 15193 raft_consensus.cc:515] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.873373 14090 master_service.cc:960] ReplaceTablet: received request to replace tablet 1c76ea235a1a4621b2bb4962c6f94656 from {username='slave'} at 127.0.0.1:33272
I20260809 01:46:39.874491 15209 leader_election.cc:290] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [CANDIDATE]: Term 1 election: Requested vote from peers 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981), 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383)
I20260809 01:46:39.882601 15184 ts_tablet_manager.cc:1434] T af393a465cb34dbdb4795df8ae8c49a4 P 9336b0d6675b4225831595357c4bdc3b: Time spent starting tablet: real 0.084s user 0.004s sys 0.003s
I20260809 01:46:39.876291 15274 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:39.888747 15274 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:39.890684 15274 leader_election.cc:290] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
I20260809 01:46:39.893236 14121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 4 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" is_pre_election: true
I20260809 01:46:39.906690 15193 leader_election.cc:290] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [CANDIDATE]: Term 1 election: Requested vote from peers 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981), e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177)
I20260809 01:46:39.877012 15002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e7e3e285bc64edb9b47e11123f2ba63" candidate_uuid: "9336b0d6675b4225831595357c4bdc3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3d5dac2381748eabc723a0c471ad03f" is_pre_election: true
I20260809 01:46:39.912019 15002 raft_consensus.cc:2393] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9336b0d6675b4225831595357c4bdc3b in current term 1: Already voted for candidate e3d5dac2381748eabc723a0c471ad03f in this term.
W20260809 01:46:39.913686 14648 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41981, user_credentials={real_user=slave}} blocked reactor thread for 59839.4us
I20260809 01:46:39.915287 14522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8174401df624fc8bb9fcd4e675042db" candidate_uuid: "36abc3bb1604427cb977fa2e6a6f4272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336b0d6675b4225831595357c4bdc3b"
I20260809 01:46:39.915781 14522 raft_consensus.cc:3060] T e8174401df624fc8bb9fcd4e675042db P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:46:39.920701 14889 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.65:41981, user_credentials={real_user=slave}} blocked reactor thread for 60115.5us
W20260809 01:46:39.921749 13972 leader_election.cc:336] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129): Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:39.922695 13972 leader_election.cc:304] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdec945a08924d01a5c58babffef9e31; no voters: 01fd3efb56f649edb893c33a263a0a93, 3d2c2e2b6aa54060a87795a74d36e4d4
I20260809 01:46:39.923338 14753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e7e3e285bc64edb9b47e11123f2ba63" candidate_uuid: "9336b0d6675b4225831595357c4bdc3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36abc3bb1604427cb977fa2e6a6f4272" is_pre_election: true
I20260809 01:46:39.923908 14753 raft_consensus.cc:2468] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9336b0d6675b4225831595357c4bdc3b in term 0.
I20260809 01:46:39.913369 15209 raft_consensus.cc:2804] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:39.923964 14996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8174401df624fc8bb9fcd4e675042db" candidate_uuid: "36abc3bb1604427cb977fa2e6a6f4272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3d5dac2381748eabc723a0c471ad03f"
I20260809 01:46:39.925134 14422 leader_election.cc:304] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36abc3bb1604427cb977fa2e6a6f4272, 9336b0d6675b4225831595357c4bdc3b; no voters: e3d5dac2381748eabc723a0c471ad03f
I20260809 01:46:39.925463 14996 raft_consensus.cc:3060] T e8174401df624fc8bb9fcd4e675042db P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:39.925946 15274 raft_consensus.cc:2749] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 01:46:39.926116 15192 raft_consensus.cc:2764] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 01:46:39.928843 14522 raft_consensus.cc:2468] T e8174401df624fc8bb9fcd4e675042db P 9336b0d6675b4225831595357c4bdc3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36abc3bb1604427cb977fa2e6a6f4272 in term 1.
I20260809 01:46:39.921018 14998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be44f2b9e9fa4ed9812b51e4aafddedf" candidate_uuid: "36abc3bb1604427cb977fa2e6a6f4272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3d5dac2381748eabc723a0c471ad03f" is_pre_election: true
I20260809 01:46:39.930945 14998 raft_consensus.cc:2468] T be44f2b9e9fa4ed9812b51e4aafddedf P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36abc3bb1604427cb977fa2e6a6f4272 in term 0.
I20260809 01:46:39.938272 14648 leader_election.cc:304] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36abc3bb1604427cb977fa2e6a6f4272, 9336b0d6675b4225831595357c4bdc3b; no voters:
I20260809 01:46:39.938962 15193 raft_consensus.cc:2804] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:39.917306 15281 raft_consensus.cc:515] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.944612 14522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be44f2b9e9fa4ed9812b51e4aafddedf" candidate_uuid: "36abc3bb1604427cb977fa2e6a6f4272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336b0d6675b4225831595357c4bdc3b"
I20260809 01:46:39.944810 15281 leader_election.cc:290] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [CANDIDATE]: Term 1 election: Requested vote from peers 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981), e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177)
I20260809 01:46:39.931573 15002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8174401df624fc8bb9fcd4e675042db" candidate_uuid: "36abc3bb1604427cb977fa2e6a6f4272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3d5dac2381748eabc723a0c471ad03f" is_pre_election: true
I20260809 01:46:39.929282 15209 raft_consensus.cc:697] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [term 1 LEADER]: Becoming Leader. State: Replica: e3d5dac2381748eabc723a0c471ad03f, State: Running, Role: LEADER
I20260809 01:46:39.945124 14522 raft_consensus.cc:3060] T be44f2b9e9fa4ed9812b51e4aafddedf P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:39.946355 15209 consensus_queue.cc:237] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.947237 14998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be44f2b9e9fa4ed9812b51e4aafddedf" candidate_uuid: "36abc3bb1604427cb977fa2e6a6f4272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3d5dac2381748eabc723a0c471ad03f"
I20260809 01:46:39.947728 14998 raft_consensus.cc:3060] T be44f2b9e9fa4ed9812b51e4aafddedf P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:39.951710 14522 raft_consensus.cc:2468] T be44f2b9e9fa4ed9812b51e4aafddedf P 9336b0d6675b4225831595357c4bdc3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36abc3bb1604427cb977fa2e6a6f4272 in term 1.
I20260809 01:46:39.952711 14648 leader_election.cc:304] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36abc3bb1604427cb977fa2e6a6f4272, 9336b0d6675b4225831595357c4bdc3b; no voters:
I20260809 01:46:39.953634 15281 raft_consensus.cc:2804] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:39.954099 15281 raft_consensus.cc:697] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [term 1 LEADER]: Becoming Leader. State: Replica: 36abc3bb1604427cb977fa2e6a6f4272, State: Running, Role: LEADER
I20260809 01:46:39.957590 15281 consensus_queue.cc:237] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
W20260809 01:46:39.961494 14888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.8.66:40383, user_credentials={real_user=slave}} blocked reactor thread for 157430us
I20260809 01:46:39.961969 15193 raft_consensus.cc:697] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [term 1 LEADER]: Becoming Leader. State: Replica: 36abc3bb1604427cb977fa2e6a6f4272, State: Running, Role: LEADER
@ 0x7f81066696db start_thread at ??:0
I20260809 01:46:39.963104 14753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e7e3e285bc64edb9b47e11123f2ba63" candidate_uuid: "e3d5dac2381748eabc723a0c471ad03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36abc3bb1604427cb977fa2e6a6f4272"
I20260809 01:46:39.963652 14753 raft_consensus.cc:3060] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:39.965566 15193 consensus_queue.cc:237] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:39.973925 14998 raft_consensus.cc:2468] T be44f2b9e9fa4ed9812b51e4aafddedf P e3d5dac2381748eabc723a0c471ad03f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36abc3bb1604427cb977fa2e6a6f4272 in term 1.
I20260809 01:46:39.975423 14753 raft_consensus.cc:2468] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3d5dac2381748eabc723a0c471ad03f in term 1.
I20260809 01:46:39.981979 14090 master_service.cc:960] ReplaceTablet: received request to replace tablet be44f2b9e9fa4ed9812b51e4aafddedf from {username='slave'} at 127.0.0.1:33272
I20260809 01:46:39.987102 15268 raft_consensus.cc:493] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:39.987603 15268 raft_consensus.cc:515] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } }
I20260809 01:46:39.990600 15268 leader_election.cc:290] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981), 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383)
I20260809 01:46:39.991644 14753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af393a465cb34dbdb4795df8ae8c49a4" candidate_uuid: "e3d5dac2381748eabc723a0c471ad03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36abc3bb1604427cb977fa2e6a6f4272" is_pre_election: true
I20260809 01:46:39.992172 14753 raft_consensus.cc:2468] T af393a465cb34dbdb4795df8ae8c49a4 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3d5dac2381748eabc723a0c471ad03f in term 0.
I20260809 01:46:39.993462 14888 leader_election.cc:304] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36abc3bb1604427cb977fa2e6a6f4272, e3d5dac2381748eabc723a0c471ad03f; no voters:
I20260809 01:46:39.994453 14522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af393a465cb34dbdb4795df8ae8c49a4" candidate_uuid: "e3d5dac2381748eabc723a0c471ad03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336b0d6675b4225831595357c4bdc3b" is_pre_election: true
I20260809 01:46:39.994962 14522 raft_consensus.cc:2468] T af393a465cb34dbdb4795df8ae8c49a4 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3d5dac2381748eabc723a0c471ad03f in term 0.
I20260809 01:46:39.995959 15268 raft_consensus.cc:2804] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:39.996253 15268 raft_consensus.cc:493] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:39.996539 15268 raft_consensus.cc:3060] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:46:40.003809 14095 catalog_manager.cc:5552] Ignoring report from unknown tablet e8174401df624fc8bb9fcd4e675042db
I20260809 01:46:40.014755 14996 raft_consensus.cc:2468] T e8174401df624fc8bb9fcd4e675042db P e3d5dac2381748eabc723a0c471ad03f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36abc3bb1604427cb977fa2e6a6f4272 in term 1.
I20260809 01:46:40.010378 15273 raft_consensus.cc:515] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } }
I20260809 01:46:40.027341 14753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c76ea235a1a4621b2bb4962c6f94656" candidate_uuid: "e3d5dac2381748eabc723a0c471ad03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36abc3bb1604427cb977fa2e6a6f4272"
I20260809 01:46:40.028010 14753 raft_consensus.cc:3060] T 1c76ea235a1a4621b2bb4962c6f94656 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Advancing to term 1
@ 0x7f81019c461f clone at ??:0
I20260809 01:46:40.037352 14753 raft_consensus.cc:2468] T 1c76ea235a1a4621b2bb4962c6f94656 P 36abc3bb1604427cb977fa2e6a6f4272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3d5dac2381748eabc723a0c471ad03f in term 1.
I20260809 01:46:40.038718 14888 leader_election.cc:304] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36abc3bb1604427cb977fa2e6a6f4272, e3d5dac2381748eabc723a0c471ad03f; no voters:
I20260809 01:46:40.040117 15282 raft_consensus.cc:2804] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:40.057279 15282 raft_consensus.cc:697] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [term 1 LEADER]: Becoming Leader. State: Replica: e3d5dac2381748eabc723a0c471ad03f, State: Running, Role: LEADER
I20260809 01:46:40.058404 15282 consensus_queue.cc:237] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } }
I20260809 01:46:40.073498 14522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c76ea235a1a4621b2bb4962c6f94656" candidate_uuid: "e3d5dac2381748eabc723a0c471ad03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336b0d6675b4225831595357c4bdc3b"
I20260809 01:46:40.074211 14522 raft_consensus.cc:3060] T 1c76ea235a1a4621b2bb4962c6f94656 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:40.081635 15273 leader_election.cc:290] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [CANDIDATE]: Term 1 election: Requested vote from peers 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383), 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981)
I20260809 01:46:40.083546 14522 raft_consensus.cc:2468] T 1c76ea235a1a4621b2bb4962c6f94656 P 9336b0d6675b4225831595357c4bdc3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3d5dac2381748eabc723a0c471ad03f in term 1.
I20260809 01:46:40.104499 15268 raft_consensus.cc:515] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } }
I20260809 01:46:40.123535 14753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af393a465cb34dbdb4795df8ae8c49a4" candidate_uuid: "e3d5dac2381748eabc723a0c471ad03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36abc3bb1604427cb977fa2e6a6f4272"
I20260809 01:46:40.124190 14753 raft_consensus.cc:3060] T af393a465cb34dbdb4795df8ae8c49a4 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:40.133106 14753 raft_consensus.cc:2468] T af393a465cb34dbdb4795df8ae8c49a4 P 36abc3bb1604427cb977fa2e6a6f4272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3d5dac2381748eabc723a0c471ad03f in term 1.
I20260809 01:46:40.137956 15268 leader_election.cc:290] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [CANDIDATE]: Term 1 election: Requested vote from peers 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981), 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383)
I20260809 01:46:40.139565 14888 leader_election.cc:304] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36abc3bb1604427cb977fa2e6a6f4272, e3d5dac2381748eabc723a0c471ad03f; no voters:
I20260809 01:46:40.140707 15268 raft_consensus.cc:2804] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:40.145413 15268 raft_consensus.cc:697] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [term 1 LEADER]: Becoming Leader. State: Replica: e3d5dac2381748eabc723a0c471ad03f, State: Running, Role: LEADER
I20260809 01:46:40.148579 15268 consensus_queue.cc:237] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } }
I20260809 01:46:40.150710 14522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af393a465cb34dbdb4795df8ae8c49a4" candidate_uuid: "e3d5dac2381748eabc723a0c471ad03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336b0d6675b4225831595357c4bdc3b"
I20260809 01:46:40.152074 14522 raft_consensus.cc:3060] T af393a465cb34dbdb4795df8ae8c49a4 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:40.187618 14522 raft_consensus.cc:2468] T af393a465cb34dbdb4795df8ae8c49a4 P 9336b0d6675b4225831595357c4bdc3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3d5dac2381748eabc723a0c471ad03f in term 1.
I20260809 01:46:40.524719 15313 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:40.529642 15313 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:40.538769 15313 leader_election.cc:290] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
W20260809 01:46:40.580754 13972 leader_election.cc:336] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129): Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
W20260809 01:46:40.610354 15098 connection.cc:573] client connection to 127.10.8.125:43535 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:40.612881 15098 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.10.8.125:43535)
W20260809 01:46:40.613658 15098 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.8.125:43535)
W20260809 01:46:40.616828 15098 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.10.8.125:43535)
W20260809 01:46:40.619236 15098 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.10.8.125:43535)
W20260809 01:46:40.621758 14889 connection.cc:573] client connection to 127.10.8.125:43535 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:40.622828 14424 connection.cc:573] client connection to 127.10.8.125:43535 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:40.626232 13975 connection.cc:573] client connection to 127.10.8.125:43535 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:40.627220 13973 connection.cc:573] server connection from 127.0.0.1:39734 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:40.629288 15317 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 01:46:40.620784 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.10.8.125:43535 (local address 127.0.0.1:53286)
0809 01:46:40.626996 (+ 6212us) negotiation.cc:107] Waiting for socket to connect
0809 01:46:40.627230 (+ 234us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":5606,"threads_started":1}
W20260809 01:46:40.642050 13975 leader_election.cc:336] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:46:40.642535 13975 leader_election.cc:304] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdec945a08924d01a5c58babffef9e31; no voters: 01fd3efb56f649edb893c33a263a0a93, 3d2c2e2b6aa54060a87795a74d36e4d4
I20260809 01:46:40.643390 15313 raft_consensus.cc:2749] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 01:46:40.777848 15319 consensus_queue.cc:1048] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:46:40.833127 15321 consensus_queue.cc:1048] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:46:40.856726 15319 consensus_queue.cc:1048] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:46:40.916774 15321 consensus_queue.cc:1048] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:46:41.025780 15346 consensus_queue.cc:1048] T be44f2b9e9fa4ed9812b51e4aafddedf P 36abc3bb1604427cb977fa2e6a6f4272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 01:46:41.031778 14648 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.8.65:41981, user_credentials={real_user=slave}} blocked reactor thread for 93605.1us
I20260809 01:46:41.072160 15346 consensus_queue.cc:1048] T e8174401df624fc8bb9fcd4e675042db P 36abc3bb1604427cb977fa2e6a6f4272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 01:46:41.142516 15347 log.cc:927] Time spent T be44f2b9e9fa4ed9812b51e4aafddedf P e3d5dac2381748eabc723a0c471ad03f: Append to log took a long time: real 0.053s user 0.001s sys 0.000s
I20260809 01:46:41.209240 15329 consensus_queue.cc:1048] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:46:41.234328 15329 consensus_queue.cc:1048] T 1c76ea235a1a4621b2bb4962c6f94656 P e3d5dac2381748eabc723a0c471ad03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:46:41.244521 15319 consensus_queue.cc:1048] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:46:41.257552 15319 consensus_queue.cc:1048] T af393a465cb34dbdb4795df8ae8c49a4 P e3d5dac2381748eabc723a0c471ad03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 01:46:41.292273 15371 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:41.293807 15371 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:41.297848 15371 leader_election.cc:290] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
W20260809 01:46:41.360632 13975 leader_election.cc:336] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:46:41.363960 13972 leader_election.cc:336] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129): Network error: Client connection negotiation failed: client connection to 127.10.8.124:34129: connect: Connection refused (error 111)
I20260809 01:46:41.364748 13972 leader_election.cc:304] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdec945a08924d01a5c58babffef9e31; no voters: 01fd3efb56f649edb893c33a263a0a93, 3d2c2e2b6aa54060a87795a74d36e4d4
I20260809 01:46:41.366119 15371 raft_consensus.cc:2749] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260809 01:46:41.208200 15258 debug-util.cc:398] Leaking SignalData structure 0x7b080002ee60 after lost signal to thread 15154
W20260809 01:46:41.805274 15154 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 1.953s user 0.641s sys 1.034s
W20260809 01:46:39.853652 15278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:41.805670 15154 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 1.954s user 0.641s sys 1.034s
I20260809 01:46:41.818058 15154 server_base.cc:1061] running on GCE node
W20260809 01:46:41.819459 15377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:46:41.820691 15154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:46:41.823240 15154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:46:41.824630 15154 hybrid_clock.cc:648] HybridClock initialized: now 1786240001824595 us; error 29 us; skew 500 ppm
I20260809 01:46:41.825474 15154 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:46:41.831717 15154 webserver.cc:533] Webserver started at http://127.10.8.124:44759/ using document root <none> and password file <none>
I20260809 01:46:41.832752 15154 fs_manager.cc:362] Metadata directory not provided
I20260809 01:46:41.833020 15154 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:46:41.842432 15154 fs_manager.cc:714] Time spent opening directory manager: real 0.006s user 0.008s sys 0.000s
I20260809 01:46:41.847299 15386 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:46:41.848505 15154 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260809 01:46:41.848879 15154 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data,/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal
uuid: "01fd3efb56f649edb893c33a263a0a93"
format_stamp: "Formatted at 2026-08-09 01:46:28 on dist-test-slave-zr1t"
I20260809 01:46:41.851161 15154 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:46:41.915161 15154 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 01:46:41.916718 15154 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:46:41.917261 15154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:46:42.011389 15154 rpc_server.cc:307] RPC server started. Bound to: 127.10.8.124:34129
I20260809 01:46:42.011446 15465 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.8.124:34129 every 8 connection(s)
I20260809 01:46:42.014222 15154 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data/info.pb
I20260809 01:46:42.023064 10273 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu as pid 15154
I20260809 01:46:42.039839 15468 sys_catalog.cc:263] Verifying existing consensus state
I20260809 01:46:42.051923 15468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: Bootstrap starting.
I20260809 01:46:42.059952 15075 heartbeater.cc:344] Connected to a master server at 127.10.8.124:34129
I20260809 01:46:42.060477 15075 heartbeater.cc:499] Master 127.10.8.124:34129 was elected leader, sending a full tablet report...
I20260809 01:46:42.063699 14831 heartbeater.cc:344] Connected to a master server at 127.10.8.124:34129
I20260809 01:46:42.064191 14831 heartbeater.cc:499] Master 127.10.8.124:34129 was elected leader, sending a full tablet report...
I20260809 01:46:42.070587 15468 log.cc:826] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: Log is configured to *not* fsync() on all Append() calls
I20260809 01:46:42.085908 15468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 01:46:42.086593 15468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: Bootstrap complete.
I20260809 01:46:42.105001 15468 raft_consensus.cc:359] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:42.108292 15468 raft_consensus.cc:740] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01fd3efb56f649edb893c33a263a0a93, State: Initialized, Role: FOLLOWER
I20260809 01:46:42.109252 15468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:42.111991 15481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:42.112594 15481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:42.113281 15468 sys_catalog.cc:565] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:46:42.159193 15075 heartbeater.cc:499] Master 127.10.8.124:34129 was elected leader, sending a full tablet report...
I20260809 01:46:42.172578 15412 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" instance_seqno: 1786239997631953) as {username='slave'} at 127.10.8.67:53895; Asking this server to re-register.
I20260809 01:46:42.174539 15075 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:42.178238 15412 ts_manager.cc:194] Registered new tserver with Master: e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177)
I20260809 01:46:42.184335 15504 catalog_manager.cc:1295] Loaded cluster ID: a317462afe1741b5bf318d3556a05e8e
I20260809 01:46:42.184680 15504 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: loading cluster ID for follower catalog manager: success
I20260809 01:46:42.192255 15504 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: acquiring CA information for follower catalog manager: success
I20260809 01:46:42.198143 15504 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 01:46:42.185889 15154 master_runner.cc:187] Error getting master registration for 127.10.8.125:43535: Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111),
I20260809 01:46:42.212348 15154 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:46:42.310737 14592 heartbeater.cc:344] Connected to a master server at 127.10.8.124:34129
I20260809 01:46:42.311169 14592 heartbeater.cc:499] Master 127.10.8.124:34129 was elected leader, sending a full tablet report...
I20260809 01:46:42.315203 15412 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" instance_seqno: 1786239992847576) as {username='slave'} at 127.10.8.65:55889; Asking this server to re-register.
I20260809 01:46:42.316641 14592 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:42.320331 15412 ts_manager.cc:194] Registered new tserver with Master: 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981)
I20260809 01:46:42.439108 15512 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:42.439512 15512 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:42.441457 15512 leader_election.cc:290] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
W20260809 01:46:42.452476 13975 leader_election.cc:336] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:46:42.455439 15426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 4 candidate_status { last_received { term: 3 index: 18 } } ignore_live_leader: false dest_uuid: "01fd3efb56f649edb893c33a263a0a93" is_pre_election: true
I20260809 01:46:42.456429 15426 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdec945a08924d01a5c58babffef9e31 in term 2.
I20260809 01:46:42.457484 13972 leader_election.cc:304] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93, cdec945a08924d01a5c58babffef9e31; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4
I20260809 01:46:42.458177 15512 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 01:46:42.458458 15512 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d2c2e2b6aa54060a87795a74d36e4d4)
I20260809 01:46:42.458709 15512 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 3 FOLLOWER]: Advancing to term 4
I20260809 01:46:42.463236 15512 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:42.464766 15512 leader_election.cc:290] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 election: Requested vote from peers 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535), 01fd3efb56f649edb893c33a263a0a93 (127.10.8.124:34129)
I20260809 01:46:42.465628 15426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdec945a08924d01a5c58babffef9e31" candidate_term: 4 candidate_status { last_received { term: 3 index: 18 } } ignore_live_leader: false dest_uuid: "01fd3efb56f649edb893c33a263a0a93"
I20260809 01:46:42.466090 15426 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 2 FOLLOWER]: Advancing to term 4
W20260809 01:46:42.468750 13975 leader_election.cc:336] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:46:42.475018 15426 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdec945a08924d01a5c58babffef9e31 in term 4.
I20260809 01:46:42.476691 13972 leader_election.cc:304] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93, cdec945a08924d01a5c58babffef9e31; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4
I20260809 01:46:42.477468 15512 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 01:46:42.479329 15512 raft_consensus.cc:697] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [term 4 LEADER]: Becoming Leader. State: Replica: cdec945a08924d01a5c58babffef9e31, State: Running, Role: LEADER
I20260809 01:46:42.480191 15512 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 3.18, Last appended by leader: 18, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:42.488637 15517 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdec945a08924d01a5c58babffef9e31. Latest consensus state: current_term: 4 leader_uuid: "cdec945a08924d01a5c58babffef9e31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:42.489188 15517 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: This master's current role is: LEADER
I20260809 01:46:42.570158 14831 heartbeater.cc:499] Master 127.10.8.124:34129 was elected leader, sending a full tablet report...
I20260809 01:46:42.575449 15412 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" instance_seqno: 1786239995533217) as {username='slave'} at 127.10.8.66:51499; Asking this server to re-register.
I20260809 01:46:42.577224 14831 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:42.582460 15412 ts_manager.cc:194] Registered new tserver with Master: 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383)
W20260809 01:46:42.981567 13975 consensus_peers.cc:597] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 -> Peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Couldn't send request to peer 3d2c2e2b6aa54060a87795a74d36e4d4. Status: Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 01:46:43.045086 15426 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Refusing update from remote peer cdec945a08924d01a5c58babffef9e31: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 19. (index mismatch)
I20260809 01:46:43.050393 15517 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 01:46:43.092936 15531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "cdec945a08924d01a5c58babffef9e31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:43.093672 15531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:43.094792 15526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdec945a08924d01a5c58babffef9e31. Latest consensus state: current_term: 4 leader_uuid: "cdec945a08924d01a5c58babffef9e31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:43.095427 15526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:43.115618 15526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "cdec945a08924d01a5c58babffef9e31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:43.116477 15526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:43.123878 15517 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "cdec945a08924d01a5c58babffef9e31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:43.124526 15517 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31 [sys.catalog]: This master's current role is: LEADER
I20260809 01:46:43.129330 15528 mvcc.cc:204] Tried to move back new op lower bound from 7316439039949103104 to 7316439032088338432. Current Snapshot: MvccSnapshot[applied={T|T < 7316439039949103104}]
I20260809 01:46:43.137710 15519 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 01:46:43.143043 15519 catalog_manager.cc:679] Loaded metadata for table default.table_645ec611a69b474199c9493389824d89 [id=5ed9016f7a8843bdb0dab988c9968d68]
I20260809 01:46:43.144341 15519 catalog_manager.cc:679] Loaded metadata for table default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 [id=9ec275401ed84e09b74eb2b454e37c4f]
I20260809 01:46:43.145447 15519 catalog_manager.cc:679] Loaded metadata for table default.table_ed71d957cdb34b9fa97483de90b1d539 [id=f71d960253a1446eae22da13257be951]
I20260809 01:46:43.150682 15519 tablet_loader.cc:96] loaded metadata for tablet 2e7e3e285bc64edb9b47e11123f2ba63 (table default.table_ed71d957cdb34b9fa97483de90b1d539 [id=f71d960253a1446eae22da13257be951])
I20260809 01:46:43.151608 15519 tablet_loader.cc:96] loaded metadata for tablet af393a465cb34dbdb4795df8ae8c49a4 (table default.table_75dfb3d85c0d413fb1d73c3ab0b38f18 [id=9ec275401ed84e09b74eb2b454e37c4f])
I20260809 01:46:43.152882 15519 tablet_loader.cc:96] loaded metadata for tablet e37b08a4896b479d9371738ef88f00a7 (table default.table_645ec611a69b474199c9493389824d89 [id=5ed9016f7a8843bdb0dab988c9968d68])
I20260809 01:46:43.153787 15519 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 01:46:43.156678 15519 catalog_manager.cc:1295] Loaded cluster ID: a317462afe1741b5bf318d3556a05e8e
I20260809 01:46:43.156872 15519 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 01:46:43.161252 15519 catalog_manager.cc:1540] Loading token signing keys...
I20260809 01:46:43.164623 15519 catalog_manager.cc:6103] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31: Loaded TSK: 0
I20260809 01:46:43.165928 15519 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 01:46:43.170419 15519 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260809 01:46:43.175910 15519 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260809 01:46:43.249583 13993 master_service.cc:960] ReplaceTablet: received request to replace tablet be44f2b9e9fa4ed9812b51e4aafddedf from {username='slave'} at 127.0.0.1:46370
I20260809 01:46:43.249835 13990 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46370:
name: "default.Table_042c570880b14f5f9317fe65d87c7cf5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:43.253351 13991 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46370:
table { table_name: "default.Table_ed71d957cdb34b9fa97483de90b1d539" } modify_external_catalogs: true
I20260809 01:46:43.254320 13991 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46370:
table { table_name: "default.Table_ed71d957cdb34b9fa97483de90b1d539" } modify_external_catalogs: true
I20260809 01:46:43.277248 14718 tablet_service.cc:1511] Processing CreateTablet for tablet e37b08a4896b479d9371738ef88f00a7 (DEFAULT_TABLE table=default.table_645ec611a69b474199c9493389824d89 [id=5ed9016f7a8843bdb0dab988c9968d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:43.278422 14718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e37b08a4896b479d9371738ef88f00a7. 1 dirs total, 0 dirs full, 0 dirs failed
F20260809 01:46:43.280330 14379 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 4 received events: NotificationEvent(eventId=1, eventTime=1786239998, eventType=CREATE_TABLE, dbName=default, tableName=table_645ec611a69b474199c9493389824d89, message={"server":"","servicePrincipal":"","db":"default","table":"table_645ec611a69b474199c9493389824d89","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_645ec611a69b474199c9493389824d89\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786239998},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/warehouse/table_645ec611a69b474199c9493389824d89\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_645ec611a69b474199c9493389824d89\",\"kudu.table_id\":\"5ed9016f7a8843bdb0dab988c9968d68\",\"transient_lastDdlTime\":\"1786239998\",\"comment\":\"\",\"kudu.master_addresses\":\"127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"a317462afe1741b5bf318d3556a05e8e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786239998,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1786239998, eventType=CREATE_TABLE, dbName=default, tableName=table_409febef55ad4ddb8638e01e247c19fb, message={"server":"","servicePrincipal":"","db":"default","table":"table_409febef55ad4ddb8638e01e247c19fb","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_409febef55ad4ddb8638e01e247c19fb\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786239998},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/warehouse/table_409febef55ad4ddb8638e01e247c19fb\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_409febef55ad4ddb8638e01e247c19fb\",\"kudu.table_id\":\"762f864272234bad897b360c54af0519\",\"transient_lastDdlTime\":\"1786239998\",\"comment\":\"\",\"kudu.master_addresses\":\"127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"a317462afe1741b5bf318d3556a05e8e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786239998,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260809 01:46:43.294482 13993 master_service.cc:960] ReplaceTablet: received request to replace tablet be44f2b9e9fa4ed9812b51e4aafddedf from {username='slave'} at 127.0.0.1:46370
I20260809 01:46:43.298569 14497 tablet_service.cc:1511] Processing CreateTablet for tablet e37b08a4896b479d9371738ef88f00a7 (DEFAULT_TABLE table=default.table_645ec611a69b474199c9493389824d89 [id=5ed9016f7a8843bdb0dab988c9968d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:43.300081 14497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e37b08a4896b479d9371738ef88f00a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:43.314680 15551 tablet_bootstrap.cc:492] T e37b08a4896b479d9371738ef88f00a7 P 36abc3bb1604427cb977fa2e6a6f4272: Bootstrap starting.
I20260809 01:46:43.330523 15551 tablet_bootstrap.cc:654] T e37b08a4896b479d9371738ef88f00a7 P 36abc3bb1604427cb977fa2e6a6f4272: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:43.341507 13986 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46370:
table { table_name: "default.Table_645ec611a69b474199c9493389824d89" } new_table_name: "default.Table_1ee29cfe13ec46e388603a636a24eb32" modify_external_catalogs: true
I20260809 01:46:43.343282 13989 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46370:
name: "default.Table_b0eef29d161f4a1e8b63b62f0e7954d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:43.346515 13987 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46370:
name: "default.Table_1a5bb6557e7a4ffcb31eb9508c96b920"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:43.350363 13988 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46370:
name: "default.Table_5fc49430cb3041e9bfed3d29e3168bb0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 01:46:43.357290 15555 tablet_bootstrap.cc:492] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b: Bootstrap starting.
I20260809 01:46:43.358530 13984 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46370:
table { table_name: "default.Table_75dfb3d85c0d413fb1d73c3ab0b38f18" } new_table_name: "default.Table_b9994acb9eb342888a41b725ad2f0c87" modify_external_catalogs: true
I20260809 01:46:43.360097 15551 tablet_bootstrap.cc:492] T e37b08a4896b479d9371738ef88f00a7 P 36abc3bb1604427cb977fa2e6a6f4272: No bootstrap required, opened a new log
I20260809 01:46:43.359932 14969 tablet_service.cc:1511] Processing CreateTablet for tablet e37b08a4896b479d9371738ef88f00a7 (DEFAULT_TABLE table=default.table_645ec611a69b474199c9493389824d89 [id=5ed9016f7a8843bdb0dab988c9968d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:43.360532 15551 ts_tablet_manager.cc:1403] T e37b08a4896b479d9371738ef88f00a7 P 36abc3bb1604427cb977fa2e6a6f4272: Time spent bootstrapping tablet: real 0.046s user 0.006s sys 0.013s
I20260809 01:46:43.361042 14969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e37b08a4896b479d9371738ef88f00a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:43.364961 15555 tablet_bootstrap.cc:654] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b: Neither blocks nor log segments found. Creating new log.
@ 0x7fafafa80920 google::LogMessage::Flush() at ??:0
I20260809 01:46:43.374816 15551 raft_consensus.cc:359] T e37b08a4896b479d9371738ef88f00a7 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:43.375515 15551 raft_consensus.cc:385] T e37b08a4896b479d9371738ef88f00a7 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:43.375782 15551 raft_consensus.cc:740] T e37b08a4896b479d9371738ef88f00a7 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36abc3bb1604427cb977fa2e6a6f4272, State: Initialized, Role: FOLLOWER
I20260809 01:46:43.376748 15555 tablet_bootstrap.cc:492] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b: No bootstrap required, opened a new log
I20260809 01:46:43.376435 15551 consensus_queue.cc:260] T e37b08a4896b479d9371738ef88f00a7 P 36abc3bb1604427cb977fa2e6a6f4272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:43.379191 15551 ts_tablet_manager.cc:1434] T e37b08a4896b479d9371738ef88f00a7 P 36abc3bb1604427cb977fa2e6a6f4272: Time spent starting tablet: real 0.018s user 0.008s sys 0.001s
I20260809 01:46:43.383265 15560 tablet_bootstrap.cc:492] T e37b08a4896b479d9371738ef88f00a7 P e3d5dac2381748eabc723a0c471ad03f: Bootstrap starting.
I20260809 01:46:43.391054 15560 tablet_bootstrap.cc:654] T e37b08a4896b479d9371738ef88f00a7 P e3d5dac2381748eabc723a0c471ad03f: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:43.377139 15555 ts_tablet_manager.cc:1403] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.010s
I20260809 01:46:43.400620 15555 raft_consensus.cc:359] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:43.401566 15555 raft_consensus.cc:385] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:43.401939 15555 raft_consensus.cc:740] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9336b0d6675b4225831595357c4bdc3b, State: Initialized, Role: FOLLOWER
I20260809 01:46:43.402876 15555 consensus_queue.cc:260] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
@ 0x7fafafa8068f google::LogMessage::~LogMessage() at ??:0
I20260809 01:46:43.408720 15560 tablet_bootstrap.cc:492] T e37b08a4896b479d9371738ef88f00a7 P e3d5dac2381748eabc723a0c471ad03f: No bootstrap required, opened a new log
I20260809 01:46:43.409118 15560 ts_tablet_manager.cc:1403] T e37b08a4896b479d9371738ef88f00a7 P e3d5dac2381748eabc723a0c471ad03f: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.011s
I20260809 01:46:43.411816 15560 raft_consensus.cc:359] T e37b08a4896b479d9371738ef88f00a7 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:43.412415 14597 heartbeater.cc:499] Master 127.10.8.126:43137 was elected leader, sending a full tablet report...
I20260809 01:46:43.412484 15560 raft_consensus.cc:385] T e37b08a4896b479d9371738ef88f00a7 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:43.412739 15560 raft_consensus.cc:740] T e37b08a4896b479d9371738ef88f00a7 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3d5dac2381748eabc723a0c471ad03f, State: Initialized, Role: FOLLOWER
I20260809 01:46:43.417656 15560 consensus_queue.cc:260] T e37b08a4896b479d9371738ef88f00a7 P e3d5dac2381748eabc723a0c471ad03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
W20260809 01:46:43.420028 13993 catalog_manager.cc:5552] Ignoring report from unknown tablet e8174401df624fc8bb9fcd4e675042db
W20260809 01:46:43.420311 13993 catalog_manager.cc:5552] Ignoring report from unknown tablet 1c76ea235a1a4621b2bb4962c6f94656
W20260809 01:46:43.420524 13993 catalog_manager.cc:5552] Ignoring report from unknown tablet be44f2b9e9fa4ed9812b51e4aafddedf
I20260809 01:46:43.420960 15555 ts_tablet_manager.cc:1434] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b: Time spent starting tablet: real 0.023s user 0.006s sys 0.005s
I20260809 01:46:43.422464 15076 heartbeater.cc:499] Master 127.10.8.126:43137 was elected leader, sending a full tablet report...
W20260809 01:46:43.428629 13983 catalog_manager.cc:5552] Ignoring report from unknown tablet e8174401df624fc8bb9fcd4e675042db
W20260809 01:46:43.428908 13983 catalog_manager.cc:5552] Ignoring report from unknown tablet 1c76ea235a1a4621b2bb4962c6f94656
W20260809 01:46:43.429100 13983 catalog_manager.cc:5552] Ignoring report from unknown tablet be44f2b9e9fa4ed9812b51e4aafddedf
I20260809 01:46:43.425237 13993 catalog_manager.cc:5719] T af393a465cb34dbdb4795df8ae8c49a4 P 9336b0d6675b4225831595357c4bdc3b reported cstate change: term changed from 0 to 1, leader changed from <none> to e3d5dac2381748eabc723a0c471ad03f (127.10.8.67). New cstate: current_term: 1 leader_uuid: "e3d5dac2381748eabc723a0c471ad03f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } }
I20260809 01:46:43.430593 15560 ts_tablet_manager.cc:1434] T e37b08a4896b479d9371738ef88f00a7 P e3d5dac2381748eabc723a0c471ad03f: Time spent starting tablet: real 0.021s user 0.003s sys 0.005s
I20260809 01:46:43.431008 13993 catalog_manager.cc:5719] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b reported cstate change: term changed from 0 to 1, leader changed from <none> to e3d5dac2381748eabc723a0c471ad03f (127.10.8.67). New cstate: current_term: 1 leader_uuid: "e3d5dac2381748eabc723a0c471ad03f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } } }
I20260809 01:46:43.437716 15563 raft_consensus.cc:493] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:43.438125 15563 raft_consensus.cc:515] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:43.440630 15563 leader_election.cc:290] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383), e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177)
I20260809 01:46:43.444803 14753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e37b08a4896b479d9371738ef88f00a7" candidate_uuid: "9336b0d6675b4225831595357c4bdc3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36abc3bb1604427cb977fa2e6a6f4272" is_pre_election: true
I20260809 01:46:43.445390 14753 raft_consensus.cc:2468] T e37b08a4896b479d9371738ef88f00a7 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9336b0d6675b4225831595357c4bdc3b in term 0.
I20260809 01:46:43.446439 14996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e37b08a4896b479d9371738ef88f00a7" candidate_uuid: "9336b0d6675b4225831595357c4bdc3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3d5dac2381748eabc723a0c471ad03f" is_pre_election: true
I20260809 01:46:43.446966 14996 raft_consensus.cc:2468] T e37b08a4896b479d9371738ef88f00a7 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9336b0d6675b4225831595357c4bdc3b in term 0.
I20260809 01:46:43.448853 14421 leader_election.cc:304] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9336b0d6675b4225831595357c4bdc3b, e3d5dac2381748eabc723a0c471ad03f; no voters:
@ 0x7fafbac00fbc kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260809 01:46:43.451716 15563 raft_consensus.cc:2804] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:43.452051 15563 raft_consensus.cc:493] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:43.452359 15563 raft_consensus.cc:3060] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:43.459060 15563 raft_consensus.cc:515] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:43.460887 15563 leader_election.cc:290] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [CANDIDATE]: Term 1 election: Requested vote from peers 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383), e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177)
I20260809 01:46:43.464015 14996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e37b08a4896b479d9371738ef88f00a7" candidate_uuid: "9336b0d6675b4225831595357c4bdc3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3d5dac2381748eabc723a0c471ad03f"
I20260809 01:46:43.464519 14996 raft_consensus.cc:3060] T e37b08a4896b479d9371738ef88f00a7 P e3d5dac2381748eabc723a0c471ad03f [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:43.465843 14753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e37b08a4896b479d9371738ef88f00a7" candidate_uuid: "9336b0d6675b4225831595357c4bdc3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36abc3bb1604427cb977fa2e6a6f4272"
I20260809 01:46:43.466368 14753 raft_consensus.cc:3060] T e37b08a4896b479d9371738ef88f00a7 P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Advancing to term 1
@ 0x7fafbac0077c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260809 01:46:43.474983 14996 raft_consensus.cc:2468] T e37b08a4896b479d9371738ef88f00a7 P e3d5dac2381748eabc723a0c471ad03f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9336b0d6675b4225831595357c4bdc3b in term 1.
I20260809 01:46:43.476011 14421 leader_election.cc:304] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9336b0d6675b4225831595357c4bdc3b, e3d5dac2381748eabc723a0c471ad03f; no voters:
I20260809 01:46:43.476853 15563 raft_consensus.cc:2804] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [term 1 FOLLOWER]: Leader election won for term 1
W20260809 01:46:43.482997 13975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111) [suppressed 8 similar messages]
I20260809 01:46:43.486168 15563 raft_consensus.cc:697] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [term 1 LEADER]: Becoming Leader. State: Replica: 9336b0d6675b4225831595357c4bdc3b, State: Running, Role: LEADER
I20260809 01:46:43.487509 15563 consensus_queue.cc:237] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:43.491963 14753 raft_consensus.cc:2468] T e37b08a4896b479d9371738ef88f00a7 P 36abc3bb1604427cb977fa2e6a6f4272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9336b0d6675b4225831595357c4bdc3b in term 1.
@ 0x7fafbac047f2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260809 01:46:43.519238 13993 catalog_manager.cc:5719] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9336b0d6675b4225831595357c4bdc3b (127.10.8.65). New cstate: current_term: 1 leader_uuid: "9336b0d6675b4225831595357c4bdc3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } health_report { overall_health: UNKNOWN } } }
@ 0x7fafbac047aa _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260809 01:46:43.529868 13983 master_service.cc:960] ReplaceTablet: received request to replace tablet e8174401df624fc8bb9fcd4e675042db from {username='slave'} at 127.0.0.1:46370
@ 0x7fafbac0473a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
@ 0x7fafbac04702 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260809 01:46:43.570147 13993 master_service.cc:960] ReplaceTablet: received request to replace tablet 2e7e3e285bc64edb9b47e11123f2ba63 from {username='slave'} at 127.0.0.1:46370
@ 0x7fafbac039fe std::__1::__function::__func<>::operator()() at ??:0
@ 0x5bd8c5 std::__1::__function::__value_func<>::operator()() at ??:?
I20260809 01:46:43.589236 13993 catalog_manager.cc:6006] T 00000000000000000000000000000000 P cdec945a08924d01a5c58babffef9e31: Sending DeleteTablet for 3 replicas of tablet 2e7e3e285bc64edb9b47e11123f2ba63
@ 0x5bd6fa std::__1::function<>::operator()() at ??:?
I20260809 01:46:43.591991 13993 catalog_manager.cc:6807] ReplaceTablet: tablet 2e7e3e285bc64edb9b47e11123f2ba63 deleted and replaced by tablet dcbb47d4375e4a1f95b2444127ea8a5e
I20260809 01:46:43.595191 14969 tablet_service.cc:1558] Processing DeleteTablet for tablet 2e7e3e285bc64edb9b47e11123f2ba63 with delete_type TABLET_DATA_DELETED (replaced by tablet dcbb47d4375e4a1f95b2444127ea8a5e) from {username='slave'} at 127.0.0.1:54934
I20260809 01:46:43.595711 14497 tablet_service.cc:1558] Processing DeleteTablet for tablet 2e7e3e285bc64edb9b47e11123f2ba63 with delete_type TABLET_DATA_DELETED (replaced by tablet dcbb47d4375e4a1f95b2444127ea8a5e) from {username='slave'} at 127.0.0.1:36920
I20260809 01:46:43.597961 14718 tablet_service.cc:1558] Processing DeleteTablet for tablet 2e7e3e285bc64edb9b47e11123f2ba63 with delete_type TABLET_DATA_DELETED (replaced by tablet dcbb47d4375e4a1f95b2444127ea8a5e) from {username='slave'} at 127.0.0.1:59400
I20260809 01:46:43.606567 15580 tablet_replica.cc:333] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b: stopping tablet replica
I20260809 01:46:43.607426 15580 raft_consensus.cc:2243] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:43.607967 15580 raft_consensus.cc:2272] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:43.610154 15580 ts_tablet_manager.cc:1916] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:43.613445 15579 tablet_replica.cc:333] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f: stopping tablet replica
I20260809 01:46:43.614290 15579 raft_consensus.cc:2243] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:46:43.615456 15579 raft_consensus.cc:2272] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:43.617588 15579 ts_tablet_manager.cc:1916] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7fafb1c6a1eb kudu::Thread::SuperviseThread() at ??:0
@ 0x4568e0 __tsan_thread_start_func at ??:?
@ 0x7fafb33ca6db start_thread at ??:0
I20260809 01:46:43.639955 14497 tablet_service.cc:1511] Processing CreateTablet for tablet dcbb47d4375e4a1f95b2444127ea8a5e (DEFAULT_TABLE table=default.table_ed71d957cdb34b9fa97483de90b1d539 [id=f71d960253a1446eae22da13257be951]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:43.641045 14497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcbb47d4375e4a1f95b2444127ea8a5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:43.649197 14717 tablet_service.cc:1511] Processing CreateTablet for tablet dcbb47d4375e4a1f95b2444127ea8a5e (DEFAULT_TABLE table=default.table_ed71d957cdb34b9fa97483de90b1d539 [id=f71d960253a1446eae22da13257be951]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:46:43.650729 14717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcbb47d4375e4a1f95b2444127ea8a5e. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fafae72561f clone at ??:0
I20260809 01:46:43.655932 15580 ts_tablet_manager.cc:1929] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:46:43.656311 15580 log.cc:1199] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-0/wal/wals/2e7e3e285bc64edb9b47e11123f2ba63
I20260809 01:46:43.657243 15580 ts_tablet_manager.cc:1950] T 2e7e3e285bc64edb9b47e11123f2ba63 P 9336b0d6675b4225831595357c4bdc3b: Deleting consensus metadata
I20260809 01:46:43.657349 15579 ts_tablet_manager.cc:1929] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:46:43.657737 15579 log.cc:1199] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-2/wal/wals/2e7e3e285bc64edb9b47e11123f2ba63
I20260809 01:46:43.658504 15579 ts_tablet_manager.cc:1950] T 2e7e3e285bc64edb9b47e11123f2ba63 P e3d5dac2381748eabc723a0c471ad03f: Deleting consensus metadata
I20260809 01:46:43.675210 15581 tablet_replica.cc:333] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272: stopping tablet replica
I20260809 01:46:43.676056 15581 raft_consensus.cc:2243] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:46:43.676674 15581 raft_consensus.cc:2272] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:46:43.678958 15581 ts_tablet_manager.cc:1916] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:46:43.696822 15555 tablet_bootstrap.cc:492] T dcbb47d4375e4a1f95b2444127ea8a5e P 9336b0d6675b4225831595357c4bdc3b: Bootstrap starting.
I20260809 01:46:43.698540 15551 tablet_bootstrap.cc:492] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272: Bootstrap starting.
I20260809 01:46:43.703783 15555 tablet_bootstrap.cc:654] T dcbb47d4375e4a1f95b2444127ea8a5e P 9336b0d6675b4225831595357c4bdc3b: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:43.704813 15551 tablet_bootstrap.cc:654] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272: Neither blocks nor log segments found. Creating new log.
I20260809 01:46:43.716581 15555 tablet_bootstrap.cc:492] T dcbb47d4375e4a1f95b2444127ea8a5e P 9336b0d6675b4225831595357c4bdc3b: No bootstrap required, opened a new log
I20260809 01:46:43.717765 15555 ts_tablet_manager.cc:1403] T dcbb47d4375e4a1f95b2444127ea8a5e P 9336b0d6675b4225831595357c4bdc3b: Time spent bootstrapping tablet: real 0.021s user 0.010s sys 0.002s
I20260809 01:46:43.719458 15551 tablet_bootstrap.cc:492] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272: No bootstrap required, opened a new log
I20260809 01:46:43.719843 15551 ts_tablet_manager.cc:1403] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.001s
I20260809 01:46:43.720433 15555 raft_consensus.cc:359] T dcbb47d4375e4a1f95b2444127ea8a5e P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:43.721092 15555 raft_consensus.cc:385] T dcbb47d4375e4a1f95b2444127ea8a5e P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:43.721360 15555 raft_consensus.cc:740] T dcbb47d4375e4a1f95b2444127ea8a5e P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9336b0d6675b4225831595357c4bdc3b, State: Initialized, Role: FOLLOWER
I20260809 01:46:43.722301 15555 consensus_queue.cc:260] T dcbb47d4375e4a1f95b2444127ea8a5e P 9336b0d6675b4225831595357c4bdc3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:43.723551 15551 raft_consensus.cc:359] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:43.724324 15551 raft_consensus.cc:385] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:46:43.724620 15551 raft_consensus.cc:740] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36abc3bb1604427cb977fa2e6a6f4272, State: Initialized, Role: FOLLOWER
I20260809 01:46:43.726920 15555 ts_tablet_manager.cc:1434] T dcbb47d4375e4a1f95b2444127ea8a5e P 9336b0d6675b4225831595357c4bdc3b: Time spent starting tablet: real 0.009s user 0.003s sys 0.003s
I20260809 01:46:43.741672 15551 consensus_queue.cc:260] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:43.746104 14832 heartbeater.cc:499] Master 127.10.8.126:43137 was elected leader, sending a full tablet report...
I20260809 01:46:43.750018 15551 ts_tablet_manager.cc:1434] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272: Time spent starting tablet: real 0.030s user 0.003s sys 0.006s
I20260809 01:46:43.758417 15581 ts_tablet_manager.cc:1929] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 01:46:43.758770 15581 log.cc:1199] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272: Deleting WAL directory at /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/ts-1/wal/wals/2e7e3e285bc64edb9b47e11123f2ba63
I20260809 01:46:43.759697 15581 ts_tablet_manager.cc:1950] T 2e7e3e285bc64edb9b47e11123f2ba63 P 36abc3bb1604427cb977fa2e6a6f4272: Deleting consensus metadata
I20260809 01:46:43.819816 15522 raft_consensus.cc:493] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:43.820333 15522 raft_consensus.cc:515] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:43.822590 15522 leader_election.cc:290] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981), e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177)
I20260809 01:46:43.824349 14522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbb47d4375e4a1f95b2444127ea8a5e" candidate_uuid: "36abc3bb1604427cb977fa2e6a6f4272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336b0d6675b4225831595357c4bdc3b" is_pre_election: true
I20260809 01:46:43.824967 14522 raft_consensus.cc:2468] T dcbb47d4375e4a1f95b2444127ea8a5e P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36abc3bb1604427cb977fa2e6a6f4272 in term 0.
W20260809 01:46:43.828310 14645 leader_election.cc:343] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177): Not found: Tablet not found: dcbb47d4375e4a1f95b2444127ea8a5e
I20260809 01:46:43.830348 14648 leader_election.cc:304] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36abc3bb1604427cb977fa2e6a6f4272, 9336b0d6675b4225831595357c4bdc3b; no voters: e3d5dac2381748eabc723a0c471ad03f
I20260809 01:46:43.831456 15522 raft_consensus.cc:2804] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:46:43.831794 15522 raft_consensus.cc:493] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:46:43.832140 15522 raft_consensus.cc:3060] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:46:43.841114 15522 raft_consensus.cc:515] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:43.843932 14522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbb47d4375e4a1f95b2444127ea8a5e" candidate_uuid: "36abc3bb1604427cb977fa2e6a6f4272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336b0d6675b4225831595357c4bdc3b"
I20260809 01:46:43.844455 14522 raft_consensus.cc:3060] T dcbb47d4375e4a1f95b2444127ea8a5e P 9336b0d6675b4225831595357c4bdc3b [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:46:43.844880 14645 leader_election.cc:343] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177): Not found: Tablet not found: dcbb47d4375e4a1f95b2444127ea8a5e
I20260809 01:46:43.845317 15522 leader_election.cc:290] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [CANDIDATE]: Term 1 election: Requested vote from peers 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981), e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177)
I20260809 01:46:43.850545 14522 raft_consensus.cc:2468] T dcbb47d4375e4a1f95b2444127ea8a5e P 9336b0d6675b4225831595357c4bdc3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36abc3bb1604427cb977fa2e6a6f4272 in term 1.
I20260809 01:46:43.851675 14648 leader_election.cc:304] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36abc3bb1604427cb977fa2e6a6f4272, 9336b0d6675b4225831595357c4bdc3b; no voters: e3d5dac2381748eabc723a0c471ad03f
I20260809 01:46:43.852394 15522 raft_consensus.cc:2804] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:46:43.853371 15522 raft_consensus.cc:697] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [term 1 LEADER]: Becoming Leader. State: Replica: 36abc3bb1604427cb977fa2e6a6f4272, State: Running, Role: LEADER
I20260809 01:46:43.854293 15522 consensus_queue.cc:237] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
W20260809 01:46:43.943881 14886 connection.cc:573] server connection from 127.0.0.1:54934 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:43.944386 15095 connection.cc:573] client connection to 127.10.8.126:43137 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:43.944753 14645 connection.cc:573] client connection to 127.10.8.126:43137 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:43.946250 15095 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.10.8.126:43137)
W20260809 01:46:43.946938 15095 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.10.8.126:43137)
W20260809 01:46:43.948045 15095 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.8.126:43137)
W20260809 01:46:43.951022 14642 connection.cc:573] server connection from 127.0.0.1:59400 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:43.951648 14421 connection.cc:573] server connection from 127.0.0.1:36920 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:43.952160 14421 connection.cc:573] client connection to 127.10.8.126:43137 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 01:46:43.955950 15095 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.10.8.126:43137)
I20260809 01:46:44.134023 15602 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cdec945a08924d01a5c58babffef9e31)
I20260809 01:46:44.134444 15602 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:44.137176 15602 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:46:44.139607 15392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:46:44.143318 15392 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:46:44.145740 15395 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:46:44.146166 15395 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:46:44.146909 15602 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:46:44.184208 10273 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu with pid 15154
W20260809 01:46:44.226658 15094 connection.cc:573] client connection to 127.10.8.124:34129 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 01:46:44.475606 15612 consensus_queue.cc:1048] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:46:44.495102 15612 consensus_queue.cc:1048] T e37b08a4896b479d9371738ef88f00a7 P 9336b0d6675b4225831595357c4bdc3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 01:46:44.818274 14645 consensus_peers.cc:597] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 -> Peer e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177): Couldn't send request to peer e3d5dac2381748eabc723a0c471ad03f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dcbb47d4375e4a1f95b2444127ea8a5e. This is attempt 1: this message will repeat every 5th retry.
I20260809 01:46:44.971957 10273 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
/tmp/dist-test-taskctLAn3/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.124:34129
--webserver_interface=127.10.8.124
--webserver_port=44759
--builtin_ntp_servers=127.10.8.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--hive_metastore_uris=thrift://127.0.0.1:35729
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260809 01:46:45.024245 15522 consensus_queue.cc:1048] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 01:46:45.327459 15628 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:46:45.328090 15628 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:46:45.328573 15628 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 01:46:45.362692 15628 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260809 01:46:45.363044 15628 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260809 01:46:45.363432 15628 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:46:45.363845 15628 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:46:45.364260 15628 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:46:45.364638 15628 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:46:45.400578 15628 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.8.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:35729
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.8.124:34129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.8.124
--webserver_port=44759
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 01:43:22 UTC on e7f111948823
build id 13251
TSAN enabled
I20260809 01:46:45.401904 15628 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:46:45.403561 15628 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 01:46:45.415010 15655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:46:45.936744 15673 ts_tablet_manager.cc:933] T dcbb47d4375e4a1f95b2444127ea8a5e P e3d5dac2381748eabc723a0c471ad03f: Initiating tablet copy from peer 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383)
I20260809 01:46:45.950004 15673 tablet_copy_client.cc:323] T dcbb47d4375e4a1f95b2444127ea8a5e P e3d5dac2381748eabc723a0c471ad03f: tablet copy: Beginning tablet copy session from remote peer at address 127.10.8.66:40383
I20260809 01:46:45.953969 14787 tablet_copy_service.cc:140] P 36abc3bb1604427cb977fa2e6a6f4272: Received BeginTabletCopySession request for tablet dcbb47d4375e4a1f95b2444127ea8a5e from peer e3d5dac2381748eabc723a0c471ad03f ({username='slave'} at 127.10.8.67:52871)
I20260809 01:46:45.954771 14787 tablet_copy_service.cc:161] P 36abc3bb1604427cb977fa2e6a6f4272: Beginning new tablet copy session on tablet dcbb47d4375e4a1f95b2444127ea8a5e from peer e3d5dac2381748eabc723a0c471ad03f at {username='slave'} at 127.10.8.67:52871: session id = e3d5dac2381748eabc723a0c471ad03f-dcbb47d4375e4a1f95b2444127ea8a5e
I20260809 01:46:45.988981 14787 tablet_copy_source_session.cc:215] T dcbb47d4375e4a1f95b2444127ea8a5e P 36abc3bb1604427cb977fa2e6a6f4272: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 01:46:46.012668 15673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcbb47d4375e4a1f95b2444127ea8a5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:46:46.057843 15673 tablet_copy_client.cc:806] T dcbb47d4375e4a1f95b2444127ea8a5e P e3d5dac2381748eabc723a0c471ad03f: tablet copy: Starting download of 0 data blocks...
I20260809 01:46:46.058770 15673 tablet_copy_client.cc:670] T dcbb47d4375e4a1f95b2444127ea8a5e P e3d5dac2381748eabc723a0c471ad03f: tablet copy: Starting download of 1 WAL segments...
I20260809 01:46:46.065629 15673 tablet_copy_client.cc:538] T dcbb47d4375e4a1f95b2444127ea8a5e P e3d5dac2381748eabc723a0c471ad03f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 01:46:46.078987 15673 tablet_bootstrap.cc:492] T dcbb47d4375e4a1f95b2444127ea8a5e P e3d5dac2381748eabc723a0c471ad03f: Bootstrap starting.
I20260809 01:46:46.105592 15673 tablet_bootstrap.cc:492] T dcbb47d4375e4a1f95b2444127ea8a5e P e3d5dac2381748eabc723a0c471ad03f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 01:46:46.106981 15673 tablet_bootstrap.cc:492] T dcbb47d4375e4a1f95b2444127ea8a5e P e3d5dac2381748eabc723a0c471ad03f: Bootstrap complete.
I20260809 01:46:46.107882 15673 ts_tablet_manager.cc:1403] T dcbb47d4375e4a1f95b2444127ea8a5e P e3d5dac2381748eabc723a0c471ad03f: Time spent bootstrapping tablet: real 0.029s user 0.021s sys 0.008s
I20260809 01:46:46.112483 15673 raft_consensus.cc:359] T dcbb47d4375e4a1f95b2444127ea8a5e P e3d5dac2381748eabc723a0c471ad03f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:46.113806 15673 raft_consensus.cc:740] T dcbb47d4375e4a1f95b2444127ea8a5e P e3d5dac2381748eabc723a0c471ad03f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3d5dac2381748eabc723a0c471ad03f, State: Initialized, Role: FOLLOWER
I20260809 01:46:46.114944 15673 consensus_queue.cc:260] T dcbb47d4375e4a1f95b2444127ea8a5e P e3d5dac2381748eabc723a0c471ad03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" member_type: VOTER last_known_addr { host: "127.10.8.65" port: 41981 } } peers { permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" member_type: VOTER last_known_addr { host: "127.10.8.66" port: 40383 } } peers { permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" member_type: VOTER last_known_addr { host: "127.10.8.67" port: 38177 } }
I20260809 01:46:46.119495 15673 ts_tablet_manager.cc:1434] T dcbb47d4375e4a1f95b2444127ea8a5e P e3d5dac2381748eabc723a0c471ad03f: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260809 01:46:46.129645 14787 tablet_copy_service.cc:342] P 36abc3bb1604427cb977fa2e6a6f4272: Request end of tablet copy session e3d5dac2381748eabc723a0c471ad03f-dcbb47d4375e4a1f95b2444127ea8a5e received from {username='slave'} at 127.10.8.67:52871
I20260809 01:46:46.130242 14787 tablet_copy_service.cc:434] P 36abc3bb1604427cb977fa2e6a6f4272: ending tablet copy session e3d5dac2381748eabc723a0c471ad03f-dcbb47d4375e4a1f95b2444127ea8a5e on tablet dcbb47d4375e4a1f95b2444127ea8a5e with peer e3d5dac2381748eabc723a0c471ad03f
W20260809 01:46:45.418031 15657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:46.801815 15662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:46:46.803423 15658 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1381 milliseconds
I20260809 01:46:46.803624 15628 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260809 01:46:46.804970 15628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:46:46.807538 15628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:46:46.809070 15628 hybrid_clock.cc:648] HybridClock initialized: now 1786240006808991 us; error 84 us; skew 500 ppm
I20260809 01:46:46.810273 15628 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:46:46.814057 14996 raft_consensus.cc:1217] T dcbb47d4375e4a1f95b2444127ea8a5e P e3d5dac2381748eabc723a0c471ad03f [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20260809 01:46:46.822631 15628 webserver.cc:533] Webserver started at http://127.10.8.124:44759/ using document root <none> and password file <none>
I20260809 01:46:46.823724 15628 fs_manager.cc:362] Metadata directory not provided
I20260809 01:46:46.824002 15628 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:46:46.832162 15628 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.000s sys 0.005s
I20260809 01:46:46.837188 15705 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:46:46.838454 15628 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260809 01:46:46.838814 15628 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data,/tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal
uuid: "01fd3efb56f649edb893c33a263a0a93"
format_stamp: "Formatted at 2026-08-09 01:46:28 on dist-test-slave-zr1t"
I20260809 01:46:46.840754 15628 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:46:46.910975 15628 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 01:46:46.912561 15628 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:46:46.913043 15628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:46:46.989471 15628 rpc_server.cc:307] RPC server started. Bound to: 127.10.8.124:34129
I20260809 01:46:46.989626 15789 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.8.124:34129 every 8 connection(s)
I20260809 01:46:46.992641 15628 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskctLAn3/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786239909098392-10273-0/minicluster-data/master-2/data/info.pb
I20260809 01:46:46.997726 10273 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskctLAn3/build/tsan/bin/kudu as pid 15628
I20260809 01:46:47.004407 15790 sys_catalog.cc:263] Verifying existing consensus state
I20260809 01:46:47.010140 15790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: Bootstrap starting.
I20260809 01:46:47.035570 15790 log.cc:826] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: Log is configured to *not* fsync() on all Append() calls
I20260809 01:46:47.099110 15790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: Bootstrap replayed 1/1 log segments. Stats: ops{read=25 overwritten=0 applied=25 ignored=0} inserts{seen=14 ignored=0} mutations{seen=20 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 01:46:47.099942 15790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: Bootstrap complete.
I20260809 01:46:47.118595 15790 raft_consensus.cc:359] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:47.120604 15790 raft_consensus.cc:740] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01fd3efb56f649edb893c33a263a0a93, State: Initialized, Role: FOLLOWER
I20260809 01:46:47.121368 15790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 25, Last appended: 4.25, Last appended by leader: 25, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:47.124353 15796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } } }
I20260809 01:46:47.125171 15796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:46:47.125442 15790 sys_catalog.cc:565] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:46:47.171243 15818 catalog_manager.cc:1295] Loaded cluster ID: a317462afe1741b5bf318d3556a05e8e
I20260809 01:46:47.171738 15818 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: loading cluster ID for follower catalog manager: success
I20260809 01:46:47.180482 15818 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: acquiring CA information for follower catalog manager: success
I20260809 01:46:47.187146 15818 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 01:46:47.156456 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:46:47.193660 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:46:47.350085 15075 heartbeater.cc:344] Connected to a master server at 127.10.8.124:34129
I20260809 01:46:47.354521 14831 heartbeater.cc:344] Connected to a master server at 127.10.8.124:34129
I20260809 01:46:47.358918 15735 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e3d5dac2381748eabc723a0c471ad03f" instance_seqno: 1786239997631953) as {username='slave'} at 127.10.8.67:44617; Asking this server to re-register.
I20260809 01:46:47.360571 15733 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "36abc3bb1604427cb977fa2e6a6f4272" instance_seqno: 1786239995533217) as {username='slave'} at 127.10.8.66:46131; Asking this server to re-register.
I20260809 01:46:47.362358 15075 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:47.365123 14831 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:47.367458 15735 ts_manager.cc:194] Registered new tserver with Master: e3d5dac2381748eabc723a0c471ad03f (127.10.8.67:38177)
I20260809 01:46:47.369912 15733 ts_manager.cc:194] Registered new tserver with Master: 36abc3bb1604427cb977fa2e6a6f4272 (127.10.8.66:40383)
I20260809 01:46:47.378655 14592 heartbeater.cc:344] Connected to a master server at 127.10.8.124:34129
I20260809 01:46:47.383477 15733 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "9336b0d6675b4225831595357c4bdc3b" instance_seqno: 1786239992847576) as {username='slave'} at 127.10.8.65:40929; Asking this server to re-register.
I20260809 01:46:47.385042 14592 heartbeater.cc:461] Registering TS with master...
I20260809 01:46:47.387935 15733 ts_manager.cc:194] Registered new tserver with Master: 9336b0d6675b4225831595357c4bdc3b (127.10.8.65:41981)
I20260809 01:46:47.744866 15847 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:47.745348 15847 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:47.748127 15847 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:46:47.751587 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:46:47.755514 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:46:47.756084 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
I20260809 01:46:47.756422 15711 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:46:47.757090 15847 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:46:48.349012 15858 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:48.349411 15858 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:48.351171 15858 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:46:48.358275 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:46:48.358916 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
I20260809 01:46:48.359226 15711 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:46:48.359826 15858 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:46:49.238857 10273 master-stress-test.cc:579] Killing test thread 1/10
I20260809 01:46:49.391445 15871 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:49.391927 15871 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:49.393347 15871 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:46:49.399070 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:46:49.399518 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
I20260809 01:46:49.399947 15711 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:46:49.400518 15871 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:46:50.226661 15884 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:50.227046 15884 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:50.228564 15884 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:46:50.233317 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:46:50.234688 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
I20260809 01:46:50.235128 15711 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:46:50.235814 15884 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:46:52.131969 15913 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:52.132530 15913 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:52.134479 15913 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:46:52.139055 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:46:52.139487 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:46:52.139779 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:46:52.140300 15913 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:46:52.196065 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:46:52.196411 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:46:55.746973 15980 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:46:55.747548 15980 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:46:55.749157 15980 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:46:55.751732 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260809 01:46:55.755395 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:46:55.756026 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:46:55.756317 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:46:55.756891 15980 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:46:57.199589 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:46:57.200011 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:47:01.146975 16104 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:47:01.147727 16104 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:47:01.150650 16104 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:47:01.154851 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:47:01.163118 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:47:01.164506 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:47:01.170964 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:47:01.172154 16104 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:47:02.202900 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:47:02.203248 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:47:03.100639 16152 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:47:03.101024 16152 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:47:03.102598 16152 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:47:03.106926 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:47:03.107362 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:47:03.107681 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:47:03.108196 16152 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:47:07.206583 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:47:07.207038 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:47:08.910996 16283 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:47:08.911501 16283 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:47:08.913308 16283 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:47:08.915583 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260809 01:47:08.920873 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:47:08.922151 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:47:08.922951 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:47:08.923710 16283 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:47:12.209728 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:47:12.210132 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:47:17.214059 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:47:17.214529 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:47:22.217204 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:47:22.217610 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:47:24.943110 16668 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:47:24.943611 16668 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:47:24.945441 16668 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:47:24.947405 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:47:24.950210 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:47:24.950757 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:47:24.951166 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:47:24.951908 16668 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:47:27.220842 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:47:27.221341 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:47:32.224754 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:47:32.225175 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:47:37.228678 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:47:37.229183 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:47:38.240675 14591 heartbeater.cc:646] Failed to heartbeat to 127.10.8.125:43535 (131 consecutive failures): Network error: Failed to ping master at 127.10.8.125:43535: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:47:38.280025 16996 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:47:38.280520 16996 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:47:38.282258 16996 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:47:38.284201 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:47:38.287835 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:47:38.288615 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:47:38.288952 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:47:38.289599 16996 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 01:47:38.659346 15074 heartbeater.cc:646] Failed to heartbeat to 127.10.8.125:43535 (131 consecutive failures): Network error: Failed to ping master at 127.10.8.125:43535: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:47:38.686655 14832 heartbeater.cc:646] Failed to heartbeat to 127.10.8.126:43137 (113 consecutive failures): Network error: Failed to ping master at 127.10.8.126:43137: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
I20260809 01:47:42.232187 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:47:42.232553 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:47:44.335355 15783 debug-util.cc:398] Leaking SignalData structure 0x7b08000a8180 after lost signal to thread 15646
W20260809 01:47:44.335988 15783 debug-util.cc:398] Leaking SignalData structure 0x7b08000700c0 after lost signal to thread 15789
I20260809 01:47:47.268730 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:47:47.269166 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:47:49.487839 17248 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:47:49.488317 17248 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:47:49.490110 17248 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:47:49.492293 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:47:49.495422 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:47:49.495934 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
I20260809 01:47:49.496234 15711 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:47:49.496796 17248 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 01:47:49.680964 14586 debug-util.cc:398] Leaking SignalData structure 0x7b08000d90c0 after lost signal to thread 14387
W20260809 01:47:49.682180 14586 debug-util.cc:398] Leaking SignalData structure 0x7b0800042ea0 after lost signal to thread 14589
I20260809 01:47:52.271961 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:47:52.272377 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:47:56.321131 17402 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:47:56.321800 17402 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:47:56.323875 17402 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:47:56.326545 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:47:56.329963 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:47:56.330487 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
I20260809 01:47:56.330789 15711 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:47:56.331387 17402 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:47:57.275231 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:47:57.275723 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:48:02.278633 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:48:02.279047 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:48:07.282552 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:48:07.282984 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:48:12.285912 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:48:12.286273 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:48:15.961459 17726 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:48:15.962101 17726 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:48:15.965152 17726 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:48:15.968015 15715 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:48:15.971297 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:48:15.971745 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:48:15.972126 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:48:15.972817 17726 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:48:17.292614 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:48:17.293228 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:48:18.318161 15068 debug-util.cc:398] Leaking SignalData structure 0x7b08000d3000 after lost signal to thread 14851
W20260809 01:48:18.319135 15068 debug-util.cc:398] Leaking SignalData structure 0x7b08000dfc80 after lost signal to thread 15073
I20260809 01:48:22.296550 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:48:22.296950 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:48:27.304648 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:48:27.305300 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:48:28.390393 14823 debug-util.cc:398] Leaking SignalData structure 0x7b08000b32e0 after lost signal to thread 14612
W20260809 01:48:28.391561 14823 debug-util.cc:398] Leaking SignalData structure 0x7b0800042600 after lost signal to thread 14828
I20260809 01:48:28.531466 18024 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:48:28.532155 18024 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:48:28.534703 18024 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:48:28.539491 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:48:28.542797 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:48:28.544430 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:48:28.544868 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:48:28.545711 18024 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:48:32.308125 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:48:32.308552 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:48:37.311926 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:48:37.312425 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:48:38.709693 14597 heartbeater.cc:646] Failed to heartbeat to 127.10.8.126:43137 (232 consecutive failures): Network error: Failed to ping master at 127.10.8.126:43137: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:48:39.110517 14830 heartbeater.cc:646] Failed to heartbeat to 127.10.8.125:43535 (250 consecutive failures): Network error: Failed to ping master at 127.10.8.125:43535: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:48:39.114051 15074 heartbeater.cc:646] Failed to heartbeat to 127.10.8.125:43535 (251 consecutive failures): Network error: Failed to ping master at 127.10.8.125:43535: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:48:39.348174 18271 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:48:39.348729 18271 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:48:39.350499 18271 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:48:39.352522 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:48:39.355738 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:48:39.356388 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:48:39.356681 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:48:39.357239 18271 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 01:48:40.990468 14823 debug-util.cc:398] Leaking SignalData structure 0x7b08000ad9e0 after lost signal to thread 14612
W20260809 01:48:40.991271 14823 debug-util.cc:398] Leaking SignalData structure 0x7b080003ad20 after lost signal to thread 14828
I20260809 01:48:42.315490 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:48:42.315932 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:48:47.231508 15717 maintenance_manager.cc:419] P 01fd3efb56f649edb893c33a263a0a93: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033363
I20260809 01:48:47.313064 15716 maintenance_manager.cc:643] P 01fd3efb56f649edb893c33a263a0a93: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.080s user 0.048s sys 0.012s Metrics: {"bytes_written":9542,"cfile_init":1,"drs_written":1,"lbm_read_time_us":265,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2626,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"rows_written":14,"thread_start_us":564,"threads_started":1,"wal-append.queue_time_us":898}
I20260809 01:48:47.315763 15717 maintenance_manager.cc:419] P 01fd3efb56f649edb893c33a263a0a93: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 932 bytes on disk
I20260809 01:48:47.317894 15716 maintenance_manager.cc:643] P 01fd3efb56f649edb893c33a263a0a93: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":332,"lbm_reads_lt_1ms":4}
I20260809 01:48:47.320254 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:48:47.320689 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:48:52.323488 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:48:52.323905 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:48:52.629926 18546 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:48:52.630446 18546 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:48:52.632332 18546 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:48:52.633656 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:48:52.636777 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:48:52.637372 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:48:52.637728 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:48:52.638334 18546 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:48:57.327893 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:48:57.328400 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:48:58.112128 18708 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:48:58.112622 18708 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:48:58.114306 18708 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:48:58.116394 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:48:58.119956 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:48:58.120565 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:48:58.120901 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:48:58.121582 18708 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:49:02.331524 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:49:02.331923 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:49:07.290009 18975 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:49:07.290841 18975 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:49:07.293546 18975 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:49:07.296873 15715 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:49:07.300071 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:49:07.300568 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:49:07.300973 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:49:07.301669 18975 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:49:07.371059 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:49:07.371596 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:49:08.109040 15783 debug-util.cc:398] Leaking SignalData structure 0x7b08000a7020 after lost signal to thread 15646
W20260809 01:49:08.109645 15783 debug-util.cc:398] Leaking SignalData structure 0x7b08000a4440 after lost signal to thread 15789
I20260809 01:49:12.374598 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:49:12.374998 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:49:17.378995 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:49:17.379537 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:49:17.587847 14823 debug-util.cc:398] Leaking SignalData structure 0x7b08000d7de0 after lost signal to thread 14612
W20260809 01:49:17.588863 14823 debug-util.cc:398] Leaking SignalData structure 0x7b0800082c40 after lost signal to thread 14828
I20260809 01:49:19.375075 19296 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:49:19.375615 19296 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
W20260809 01:49:19.389173 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260809 01:49:19.391458 19296 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:49:19.395711 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:49:19.396569 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:49:19.397001 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:49:19.397712 19296 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:49:20.052884 19319 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:49:20.053480 19319 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:49:20.055603 19319 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:49:20.060724 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:49:20.061194 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:49:20.061641 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:49:20.062256 19319 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 01:49:21.898051 14586 debug-util.cc:398] Leaking SignalData structure 0x7b08000816c0 after lost signal to thread 14387
W20260809 01:49:21.899142 14586 debug-util.cc:398] Leaking SignalData structure 0x7b0800080040 after lost signal to thread 14589
I20260809 01:49:22.382444 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:49:22.382818 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:49:27.386358 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:49:27.386893 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:49:29.088527 14823 debug-util.cc:398] Leaking SignalData structure 0x7b080003d780 after lost signal to thread 14612
W20260809 01:49:29.089447 14823 debug-util.cc:398] Leaking SignalData structure 0x7b08000adda0 after lost signal to thread 14828
I20260809 01:49:32.389604 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:49:32.390018 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:49:33.041499 19626 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:49:33.042141 19626 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:49:33.044565 19626 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:49:33.047176 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260809 01:49:33.050915 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:49:33.051944 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:49:33.052220 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:49:33.052785 19626 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:49:37.393733 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:49:37.394332 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:49:39.132615 14591 heartbeater.cc:646] Failed to heartbeat to 127.10.8.125:43535 (371 consecutive failures): Network error: Failed to ping master at 127.10.8.125:43535: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:49:39.521886 14830 heartbeater.cc:646] Failed to heartbeat to 127.10.8.125:43535 (370 consecutive failures): Network error: Failed to ping master at 127.10.8.125:43535: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:49:39.526500 15074 heartbeater.cc:646] Failed to heartbeat to 127.10.8.125:43535 (371 consecutive failures): Network error: Failed to ping master at 127.10.8.125:43535: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:49:42.396996 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:49:42.397408 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:49:47.401196 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:49:47.401757 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:49:51.296140 20092 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:49:51.296797 20092 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:49:51.299451 20092 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:49:51.301865 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:49:51.305621 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:49:51.306254 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:49:51.306612 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:49:51.307219 20092 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:49:52.583218 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:49:52.583668 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:49:52.698381 15783 debug-util.cc:398] Leaking SignalData structure 0x7b08000a4f40 after lost signal to thread 15646
W20260809 01:49:52.699000 15783 debug-util.cc:398] Leaking SignalData structure 0x7b08000703a0 after lost signal to thread 15789
W20260809 01:49:53.664019 15068 debug-util.cc:398] Leaking SignalData structure 0x7b08000c87a0 after lost signal to thread 14851
W20260809 01:49:53.665066 15068 debug-util.cc:398] Leaking SignalData structure 0x7b08000c7a00 after lost signal to thread 15073
I20260809 01:49:55.920853 20159 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:49:55.921562 20159 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:49:55.924085 20159 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:49:55.930676 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:49:55.931308 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:49:55.931695 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:49:55.932417 20159 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:49:57.586779 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:49:57.587268 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:50:02.590111 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:50:02.590526 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:50:07.594098 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:50:07.594612 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:50:11.428346 20466 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:50:11.428890 20466 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:50:11.431289 20466 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:50:11.434257 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260809 01:50:11.437538 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:50:11.438117 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:50:11.438462 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:50:11.439129 20466 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:50:12.597448 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:50:12.597898 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:50:13.814504 20517 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:50:13.814945 20517 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:50:13.817451 20517 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:50:13.823086 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:50:13.823525 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:50:13.823856 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:50:13.824426 20517 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:50:17.601411 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:50:17.601958 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:50:20.718748 14586 debug-util.cc:398] Leaking SignalData structure 0x7b08000d6b20 after lost signal to thread 14387
W20260809 01:50:20.720323 14586 debug-util.cc:398] Leaking SignalData structure 0x7b0800040c60 after lost signal to thread 14589
I20260809 01:50:22.604770 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:50:22.605201 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:50:27.609279 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:50:27.609952 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:50:29.218780 20846 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:50:29.219355 20846 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:50:29.221484 20846 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:50:29.224136 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260809 01:50:29.227053 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:50:29.227763 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:50:29.228037 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:50:29.228540 20846 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:50:32.612892 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:50:32.613335 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:50:37.616899 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:50:37.617404 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:50:39.572327 14597 heartbeater.cc:646] Failed to heartbeat to 127.10.8.126:43137 (472 consecutive failures): Network error: Failed to ping master at 127.10.8.126:43137: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:50:39.865387 15076 heartbeater.cc:646] Failed to heartbeat to 127.10.8.126:43137 (473 consecutive failures): Network error: Failed to ping master at 127.10.8.126:43137: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:50:39.938957 14830 heartbeater.cc:646] Failed to heartbeat to 127.10.8.125:43535 (490 consecutive failures): Network error: Failed to ping master at 127.10.8.125:43535: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:50:42.219991 21111 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:50:42.220600 21111 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:50:42.223142 21111 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:50:42.225796 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:50:42.230393 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:50:42.230988 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:50:42.231266 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:50:42.231812 21111 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:50:42.619508 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:50:42.619961 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:50:46.243400 14823 debug-util.cc:398] Leaking SignalData structure 0x7b08000ad920 after lost signal to thread 14612
W20260809 01:50:46.244318 14823 debug-util.cc:398] Leaking SignalData structure 0x7b08000b3820 after lost signal to thread 14828
I20260809 01:50:47.624009 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:50:47.624507 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:50:52.627672 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:50:52.628089 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:50:54.514248 15068 debug-util.cc:398] Leaking SignalData structure 0x7b08000d75a0 after lost signal to thread 14851
W20260809 01:50:54.515214 15068 debug-util.cc:398] Leaking SignalData structure 0x7b08000d5e60 after lost signal to thread 15073
W20260809 01:50:56.790200 15783 debug-util.cc:398] Leaking SignalData structure 0x7b08000700a0 after lost signal to thread 15646
W20260809 01:50:56.790818 15783 debug-util.cc:398] Leaking SignalData structure 0x7b08000705e0 after lost signal to thread 15789
I20260809 01:50:57.352540 21404 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:50:57.353147 21404 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:50:57.355554 21404 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:50:57.358466 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:50:57.362128 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:50:57.363059 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:50:57.363430 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:50:57.364153 21404 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:50:57.674031 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:50:57.674561 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:51:02.677350 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:51:02.677834 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:51:07.681856 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:51:07.682420 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:51:09.097201 14823 debug-util.cc:398] Leaking SignalData structure 0x7b08000cdc40 after lost signal to thread 14612
W20260809 01:51:09.098127 14823 debug-util.cc:398] Leaking SignalData structure 0x7b0800082f20 after lost signal to thread 14828
I20260809 01:51:12.686030 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:51:12.686508 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:51:14.478281 21761 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:51:14.478833 21761 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:51:14.481348 21761 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:51:14.483633 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:51:14.487061 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:51:14.487564 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
I20260809 01:51:14.487891 15711 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:51:14.488474 21761 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:51:17.690354 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:51:17.690800 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:51:21.131394 21899 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:51:21.131911 21899 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:51:21.133761 21899 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:51:21.137251 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:51:21.140682 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
W20260809 01:51:21.141177 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
I20260809 01:51:21.141489 15711 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:51:21.142134 21899 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 01:51:22.693946 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:51:22.694350 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:51:27.698172 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:51:27.698681 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 01:51:32.145463 14586 debug-util.cc:398] Leaking SignalData structure 0x7b08000b45a0 after lost signal to thread 14387
W20260809 01:51:32.146612 14586 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9a20 after lost signal to thread 14589
I20260809 01:51:32.701694 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:51:32.702126 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:51:37.705641 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:51:37.706139 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:51:37.853179 22231 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:51:37.853987 22231 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdec945a08924d01a5c58babffef9e31" member_type: VOTER last_known_addr { host: "127.10.8.126" port: 43137 } } peers { permanent_uuid: "3d2c2e2b6aa54060a87795a74d36e4d4" member_type: VOTER last_known_addr { host: "127.10.8.125" port: 43535 } } peers { permanent_uuid: "01fd3efb56f649edb893c33a263a0a93" member_type: VOTER last_known_addr { host: "127.10.8.124" port: 34129 } }
I20260809 01:51:37.860442 22231 leader_election.cc:290] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137), 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535)
W20260809 01:51:37.863438 15711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 01:51:37.867553 15711 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cdec945a08924d01a5c58babffef9e31 (127.10.8.126:43137): Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:51:37.869402 15715 leader_election.cc:336] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d2c2e2b6aa54060a87795a74d36e4d4 (127.10.8.125:43535): Network error: Client connection negotiation failed: client connection to 127.10.8.125:43535: connect: Connection refused (error 111)
I20260809 01:51:37.869801 15715 leader_election.cc:304] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01fd3efb56f649edb893c33a263a0a93; no voters: 3d2c2e2b6aa54060a87795a74d36e4d4, cdec945a08924d01a5c58babffef9e31
I20260809 01:51:37.870592 22231 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 01fd3efb56f649edb893c33a263a0a93 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 01:51:38.858276 15783 debug-util.cc:398] Leaking SignalData structure 0x7b08000a6d40 after lost signal to thread 15646
W20260809 01:51:38.858873 15783 debug-util.cc:398] Leaking SignalData structure 0x7b0800070080 after lost signal to thread 15789
W20260809 01:51:40.008249 14597 heartbeater.cc:646] Failed to heartbeat to 127.10.8.126:43137 (592 consecutive failures): Network error: Failed to ping master at 127.10.8.126:43137: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:51:40.135334 15076 heartbeater.cc:646] Failed to heartbeat to 127.10.8.126:43137 (593 consecutive failures): Network error: Failed to ping master at 127.10.8.126:43137: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:51:40.358503 14832 heartbeater.cc:646] Failed to heartbeat to 127.10.8.126:43137 (593 consecutive failures): Network error: Failed to ping master at 127.10.8.126:43137: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
I20260809 01:51:42.751950 15628 master_runner.cc:187] Error getting master registration for 127.10.8.126:43137: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111),
I20260809 01:51:42.752422 15628 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:51:43.741958 15717 maintenance_manager.cc:419] P 01fd3efb56f649edb893c33a263a0a93: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 932 bytes on disk
I20260809 01:51:43.755225 15716 maintenance_manager.cc:643] P 01fd3efb56f649edb893c33a263a0a93: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.004s sys 0.008s Metrics: {"dirs.queue_time_us":790,"dirs.run_cpu_time_us":277,"dirs.run_wall_time_us":277,"thread_start_us":926,"threads_started":2,"wal-append.queue_time_us":709}
W20260809 01:51:43.886389 15098 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:51:43.886868 15098 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:51:43.887140 15098 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
W20260809 01:51:43.887387 15098 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
F20260809 01:51:43.898783 15106 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b0eef29d161f4a1e8b63b62f0e7954d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260809 01:51:43.900391 15125 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.10.8.126:43137,127.10.8.125:43535,127.10.8.124:34129, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.8.126:43137: connect: Connection refused (error 111)
*** Aborted at 1786240303 (unix time) try "date -d @1786240303" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800002821) received by PID 10273 (TID 0x7f56bd1fe700) from PID 10273; stack trace: ***
@ 0x2b07a0 __tsan::CallUserSignalHandler() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2b2c83 rtl_sigaction() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
*** Check failure stack trace: ***
@ 0x7f56cf2cd980 (unknown) at ??:0
@ 0x7f56ca487e87 gsignal at ??:0
@ 0x7f56ca4897f1 abort at ??:0
@ 0x2b6796 __interceptor_abort at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
@ 0x7f56cd51fd74 google::LogMessage::SendToLog() at ??:0
@ 0x7f56cd520920 google::LogMessage::Flush() at ??:0
@ 0x7f56cd525a5b google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x35eb04 kudu::MasterStressTest::CreateTableThread() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:?
@ 0x38c2fe _ZNSt3__18__invokeIMN4kudu16MasterStressTestEFvvEPNS1_26MasterStressTest_Test_TestEJEvEEDTcldsdeclsr3std3__1E7forwardIT0_Efp0_Efp_spclsr3std3__1E7forwardIT1_Efp1_EEEOT_OS7_DpOS8_ at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3846
@ 0x38c1b1 _ZNSt3__116__thread_executeINS_10unique_ptrINS_15__thread_structENS_14default_deleteIS2_EEEEMN4kudu16MasterStressTestEFvvEJPNS6_26MasterStressTest_Test_TestEEJLm2EEEEvRNS_5tupleIJT_T0_DpT1_EEENS_15__tuple_indicesIJXspT2_EEEE at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/thread:281
@ 0x38b8fb std::__1::__thread_proxy<>() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/thread:?
@ 0x2b0660 __tsan_thread_start_func at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:959
@ 0x7f56cf2c26db start_thread at ??:0
@ 0x7f56ca56a61f clone at ??:0